Basic Model: From Revenue Sources to Classroom Resources

The basic model linking available revenues, schooling resources and ultimately student outcomes remains relatively simple. Setting aside federal revenue for the moment, which is about 10% of education revenue on average, Figure 12 illustrates that the amount of revenue state and local education systems tend to have is a function of both a) fiscal capacity and b) fiscal effort.  On average, some states put forth more effort than others and some states have more capacity to raise revenue than others. These differences lead to vast differences in school funding across states, and subsequently to vast differences in classroom resources.  Federal aid is far from sufficient to close these gaps. Similarly, within some states, local districts have widely varied capacity to raise revenue for their schools, and some differences in revenues also result from differences in local effort. State effort to close these gaps varies widely.

Whether at the state level on average, or at the local level, the amount of revenue available dictates directly the amount that can be spent. Current operating expenditures are balanced primarily between the quantity of staff employed, often reflected in pupil to teacher ratios or class sizes, and the compensation of those employees including salaries and benefits. Therein lies the primary tradeoff to be made in school district planning and budgeting. Yes, one could trade off teachers for computers (or “tech-based solutions”), but most schools don’t and those that do have not produced impressive results. At any given budget constraint, staffing quantities can be increased at the expense of wages, or wages increased at the expense of quantities. The combination of the two must be sufficient to achieve desired outcomes.

Figure 12

These connections matter from an equity perspective, from an adequacy perspective and from productivity and efficiency perspectives.

From an equity perspective, if revenue sources remain unequal, leading to unequal expenditures, those inequities will lead to unequal staffing ratios and/or wages, resulting in unequal school quality and eventually outcomes. The argument that schools and districts with less money simply need to use it better – figure out how to hire better teachers in each position, with lower total payroll, is an inequitable requirement. This argument is based on loosely on the popular book Moneyball, which touted the success of the Oakland Athletics, applying new metrics to recruit undervalued players. But this argument forgets the book’s subtitle “the art of winning an unfair game,” a reference to the difficulties of small market teams competing successfully in a sport which only loosely regulates payroll parity. Clearly, baseball would be a fairer game if all teams could fund equal total payroll. While it’s up to Major League Baseball to decide whether greater parity across teams – a fairer game – is preferable (and profitable), public education shouldn’t be an unfair game.

From an adequacy perspective, sufficient funding is a necessary condition for hiring sufficient quantities of qualified personnel, to achieve sufficient outcomes.  Finally, from a productivity and efficiency perspective, we do know that human resource intensive strategies tend to be productive, and, as of yet, we have little evidence of substantial efficiency gains from technological substitutions resulting in substantial reduction of human resources.  Resource constrained schools and districts which already have large class sizes and non-competitive wages are unable to trade off one for the other. Put bluntly, if you don’t have it, you can’t spend it.

Variations in Revenues and Classroom Resources

Across states, average state and local revenues are partly a function of differences in “effort” and partly a function of differences in “capacity.” [i] Figure 13 shows the relationship across states, in 2015 between our measure of “capacity” – gross domestic product (GDP state) – and total state and local revenues per pupil. Differences in state GDP explain about 42% of the differences in state and local revenue. Higher capacity states like Massachusetts, Connecticut, New York, Wyoming and Alaska tend to raise more revenue for schools. Vermont stands out as a state where capacity is below the median but revenue is very high. Mississippi stands out as a state with very low capacity and very low revenue.

Figure 13

[1] per pupil revenue projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty[ii]

Figure 14 shows the relationship between a common measure of “effort” and state and local revenues. The effort measure is the ratio of state and local revenues to gross state product. That is, what percent of economic capacity are states spending on schools? Differences in effort explain about 1/3 of differences in state and local revenue per pupil. Despite having low capacity, Vermont has very high effort which results in high revenue levels. Other higher capacity states like Connecticut, Massachusetts and New York are able to raise relatively high revenue levels with much lower effort. Mississippi by contrast leverages above average effort, but because of its very low capacity, simply can’t dig itself out. Other states like Arizona and North Carolina, which have much greater capacity than Mississippi, simply choose not to apply effort toward raising school revenues.

Figure 14

[1] per pupil revenue projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

Figure 15 displays the rather obvious relationship that, the more state and local revenue per pupil raised in a state, the more that is spent per pupil.

Figure 15

[1] per pupil revenue projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

[2] current spending per pupil projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

Figure 16 shows that greater per pupil spending generally leads to more teachers per 100 pupils. Very low spending states (also low effort) like Arizona and Nevada tend to have very low staffing ratios compared to other states.

Figure 16

[2] current spending per pupil projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

[3] teachers per 100 pupils projected for district with >2,000 pupils, average population density, average labor costs and 20% children in poverty

Figure 17 shows that states spending more per pupil, on average, tend to have more competitive teacher wages. Here, the indicator of competitive wages is the ratio of salaries of elementary and secondary teachers to same age, similarly educated non-teachers in the same labor market within each state.  That is, the competitiveness of teacher wages is relative. As indicated by decades of teacher labor market research, the relative competitiveness of teacher wages influences the quality of candidates who enter teaching as a profession.[iii] Teacher compensation is especially poor in states like Arizona, which has among the lowest per pupil spending in the nation. By contrast, teacher compensation is highly competitive in Vermont and Wyoming, both higher spending states, but also states where non-teacher compensation is not particularly high, compared to New York, New Jersey and Connecticut.

Figure 17

District Spending Determines School Resources (Illinois Examples)

Similar disparities play out at the local level, especially in those states which have done the least to resolve disparities between rich and poor districts.  Year after year, a report on which I am an author – Is School Funding Fair? – identifies Illinois and Pennsylvania as among the most disparate states in the nation. Specifically, in these two states, districts serving high poverty student populations continue to have far fewer resources per pupil than their more advantaged counterparts. Here, I use Illinois as an example of the resulting relationship between a) district level spending variation, b) school site spending variation, and c) on-the-ground resources including staffing assignments per pupil.  These connections seem obvious. If a district has sufficient revenues, those revenues can be used to support the schools in that district, leading to more and better paid teachers. If a district doesn’t have the resources, neither will the schools in that district.

I raise this issue because, over the past decade, federal policy (and the think tanks which inform federal policy) have placed disproportionate emphasis on the need to resolve inequities in spending across (particularly staffing expenditures) schools within districts, while ignoring inequities in spending between districts.  But, especially in highly inequitable states like Illinois, resolving between school inequities in poorly funded school districts is akin to re-arranging deck chairs on the Costa Concordia.[iv]

Figure 18 shows the relationship between district level operating expenditures per pupil (horizontal axis) and school site staffing expense per pupil on the vertical axis, for the Chicago metropolitan area in 2012. Only a few districts have large numbers of schools across which resources might vary. The City of Chicago schools appear at approximately the $15,000 per pupil point on the horizontal axis as a vertical pattern of circles. Circle size indicates school enrollment. While there appears to be substantial variation in resources across Chicago schools, much (about half) of that variation is a function of differences in student populations (special education in particular) and grade ranges served.

On average, Figure 18 shows that schools in districts that spend more per pupil overall, spend more per pupil at the school level on staffing. 44% of the variation in staffing expenditure across schools is driven by spending differences across districts. And about half of the remaining differences in staffing expenditure across schools are driven by the distribution of special education populations across schools and district targeting of staff to serve those children.[v]  That is, much of the within district variation that does exist is rational. Resources that flow to districts pay for staff that work in schools. It really is that simple. If the district has more revenue coming in, it can spend more, and that spending shows up in the form of more and better paid teachers in schools.

Figure 18

Figure 19 shows how, in the Chicago metropolitan area, differences in district level operating expenditures lead to differences in competitive wages for teachers.  Here, competitiveness of wages are measured in terms of the salaries of teachers with specific numbers of years of experience and degree levels (contract hours and days per year, grade level taught, and job assignment), compared to their peers in other schools in the Chicago metro area. Salary competitiveness measured in this way reflects the recruitment and retention potential of districts, for those who have already chosen to teach. Salary differentials within teaching may influence teacher sorting across schools and districts.

Figure 19 shows that teachers in low spending districts tend to have salaries at about 75% of the average teacher with similar credentials in a similar assignment. Teachers in high spending districts tend to have salaries around 25% higher than the average teacher in a similar position with similar credentials.

Figure 19

Finally, Table 1 summarizes features of Illinois school districts which I identified in a paper in 2012[vi] as falling into quadrants where the upper right (advantaged) quadrant were districts with high spending and high outcomes and the lower left (disadvantaged) were those with low spending and low outcomes. To identify advantaged and disadvantaged school districts, spending was adjusted for the costs of providing equal educational opportunity to all students by methods discussed in a forthcoming post. Nominal and adjusted spending levels are reported in Table 1.

In my analysis, there were 146 high spending high outcome elementary districts serving over 250 thousand children and 120 low spending low outcome districts serving about 150 thousand children. There were 41 high spending high outcome secondary districts serving up to 150 thousand children and 25 low spending low outcome secondary districts serving about 50 thousand children. For unified districts, there were 82 that were high spending and high outcome, serving 250 thousand children and 156 that are low spending with low outcomes, serving over 800 thousand children, with about half of those children attending Chicago Public Schools.

Even without any adjustment for costs or needs, the average per pupil operating expenditures are lower in low spending, low outcome districts. The percent of children who are low income is substantially higher in low spending, low outcome districts. Table 1 shows that the district operating expenditure advantages of the high resource – high outcome district schools translate directly to a) more staff per pupil and b) better paid staff per pupil. In 2009, high resource schools had 86 and 79 teacher assignments per 1000 pupils in elementary and secondary schools respectively, compared to 79 and 72 for low resource schools. Further, for teachers with similar training and experience in similar assignments, teachers in low resource schools were paid $7,000 to $7,500 less in salary. That is, schools in financially advantaged districts have both more staff per pupil and are able to pay them more, while serving far less needy student population.

The take-home point of this post is that on average, the financial resources available to schools generally translate to human resources. Financial resources translate to the quantities of staff which can be hired or retained and they translate to the wages that can be paid. And both of these matter, as will be discussed more extensively in future posts.  To summarize:

1. Schooling is human resource intensive, whether traditional public, charter or private;
2. When schools have more money, they invest it in more staff and better paid staff and when they don’t, they can’t!

More later on whether charter schools and private schools have much to offer in the way of technological innovation, substitution and creative resource allocation.

NOTES

[i] Baker, B. D. (2016). School Finance & the Distribution of Equal Educational Opportunity in the Postrecession US. Journal of Social Issues, 72(4), 629-655.

[ii] Baker, B.D., Srikanth, A., Weber, M.A. (2016). Rutgers Graduate School of Education/Education Law Center: School Funding Fairness Data System. Retrieved from: http://www.schoolfundingfairness.org/data-download

[iii] Baker, B. D. (2016). Does money matter in education?. Albert Shanker Institute.

[iv] Baker, B. D. (2012). Rearranging deck chairs in Dallas: Contextual constraints and within-district resource allocation in urban Texas school districts. Journal of Education Finance, 37(3), 287-315.

[v] Baker, B.D., Srikanth, A., Weber, M. (2017) The Incompatibility of Federal Policy Preferences for Charter School Expansion and Within District Equity.

[vi] Baker, B. D. (2012, March). Unpacking the consequences of disparities in school district financial inputs: Evidence from staffing data in New York and Illinois. In annual meeting of the Association for Education Finance and Policy, Boston, MA.

Over the next few months, I plan to share some rough drafts of forthcoming work – Here, I present a brief reality check on what is often referred to as the “long term trend” argument proving that money doesn’t matter for schools.  Here’s Bill Gates’ version of that argument:

Over the past four decades, the per-student cost of running our K-12 schools has more than doubled, while our student achievement has remained virtually flat. Meanwhile, other countries have raced ahead.[i]

Bill Gates, Microsoft co-founder

[i] http://www.washingtonpost.com/wp-dyn/content/article/2011/02/27/AR2011022702876.html

Reality Check

Here, I look at long term national trends in actual student outcomes and in school resources.  Recall that the need for disruptive reform and magic elixirs derives primarily from our massive increases in spending coupled with our virtually flat student outcomes over time.  But are those claims even remotely correct?

Richard Rothstein of the Economic Policy Institute immediately critiqued Bill Gates’ assertions of “virtually flat” student outcomes in a memo titled “Fact-Challenged Policy.” Rothstein (2011) shows that, in fact, “On these exams [National Assessment of Educational Progress], American students have improved substantially, in some cases phenomenally.” (p. 1)[i] Related work by Rothstein and colleague Martin Carnoy (2013) confirms that, when accounting for differences in student disadvantage, U.S students perform much better than what is suggested by commonly cited, unadjusted rankings that fail to account for changes in subgroup proportions when aggregating test results.[ii]

In 2010, Educational Testing Service (ETS) released “The Black-White Achievement Gap: When Progress Stopped,” a report by Paul Barton and Rich Coley (2010), in which the authors explored the Black-White achievement gap from the 1970s to recent years.[iii] The goal of that report was to explore trends in Black-White achievement gaps, and changing conditions which may explain those trends.  Barton and Coley explained that “From the early 1970s until the late 1980s, a very large narrowing of the gap occurred in both reading and mathematics, with the size of the reduction depending on the subject and age group examined.” (p. 7) Reductions to achievement gaps were particularly pronounced in reading among 13 and 17 year olds, while still significant in mathematics. However, “During the 1990s, the gap narrowing generally halted, and actually began to increase in some cases.” (p. 7). The authors note some additional gap narrowing from 1999 to 2004 and mixed findings from 2004 to 2008. Rothstein (2011) shows that, even during the period from 1990 to 2008, achievement gains for Black 4^th and 8^th grade students have been substantial in mathematics in particular, and have outpaced their White peers.[iv]

Figure 3 displays the long term trends for black and white children at age 13 on the NAEP Long Term Trend assessment.  Both black and white scores trend upward, and, as noted by Barton and Coley, Black scores increase significantly from the 1970s through 1990.

Figure 3

NAEP long term trends on reading and math for 13 year olds

Figure 4 shows more recent trends for children by income status for Reading and Figure 5 shows the trends for math at 4^th and 8^th grade. For reading, fourth grade scores have continued to trend upward for all groups, but for eight graders, dip slightly in 2015. For math, the overall upward trend is also consistent across grades, but also with a dip in 2015.  It would be premature to assume any causation for the 2015 dip.

Figure 4

NAEP reading by income status

Figure 5

NAEP math by income status

Figure 6 compares nominal (not inflation adjusted) current spending per pupil to current spending per pupil adjusted for the costs of maintaining competitive wages over time. The previous figure – THE GRAPH – uses a consumer price index, which is less appropriate for evaluating the value of the education dollar over time.  By 2015, the average school district nationally was roughly at a 10 year break-even point on per pupil spending. That is, per pupil spending hasn’t risen for a decade and has barely risen for over two decades (2.5%).  So, no, school spending is not dramatically increasing over time and has declined in real terms from 2009 to 2015, the most recent national district level data.

Figure 6

Current Operating Expenditures per Pupil Adjusted for Labor Costs

            Over the longer term, Figure 7 shows that direct government expenditure on elementary and secondary education as a share of gross domestic product has oscillated over decades but is presently about where it was both 15 years earlier (2000) and 40 years earlier (1975). That is, education spending is not outstripping our economic capacity to pay for it.

Figure 7

Direct Education Expense as a Share of Gross Domestic Product

Some have asserted that along with ballooning spending over time has been a staffing surge. That is, schools have hired more and more people and have especially hired more and more individuals with less direct classroom contact time – administrative and support staff, including specialists.[v] There is some truth to the fact that teaching specialist and school level administrative positions have increased over time, in part because of additional legal protections for specialized programs and services for children with disabilities.[vi]  But, as Figure 8, which includes all staff classified as teachers (specialists among them), total numbers of teachers per each 100 pupils are at a break-even point from about 12 to 14 years prior (2001 to 2003). That is, since the early 2000s (when NAEP progress seemed to slow down), teaching staff have remained relatively stagnant. There was a modest bump in the mid-2000s, which subsided during the “new normal” period and has not since rebounded. Notably, a recent comprehensive meta-analysis of interventions which improve outcomes for low income students found specifically that the most effective interventions were those involving increased human resources.[vii]

Figure 8.

Teachers (all) per 100 pupils over time

Finally, Figure 9 shows the ratio of teacher weekly wages to those of college educated non-teachers from 1979 to 2015, estimated by authors from the Economic Policy Institute. For the past 15 years, teacher wages have held constant at about 77% of non-teacher wages. Some assert that teacher’s healthy and growing pension benefits are a substantial offset to this gap, noting that teacher benefits as a share of teacher wages have increased from about 10% to about 20% of wages since the early 2000s, whereas private sector benefits have held constant at 10% wages. [viii] One problem is that the difference is expressed in percent terms, and the supposed 20% rate for teachers is over a wage that is 23% lower to begin with.  Of course, it’s a larger and growing percent of a lower and declining wage. Even taking that percent at face value as correct, would at best raise teacher wages to about 84% of non-teacher wages in 2015.[ix]

Figure 9

Ratio of teacher weekly wages to college educated non-teacher weekly wages

To summarize:

• NAEP scores have increased substantively over the long term, but have slowed in growth more recently, as has closure of racial achievement gaps;
• School spending has been relatively stagnant for the past decade and so too have staffing quantities and the competitiveness of teacher wages;
• Over the shorter term, since 2008, spending, staffing and wage competitiveness have all declined.

These long term trends are roughly the opposite of those used most often to proclaim the failures of American education.  One implication of these trends is that U.S. public schools have, in fact, become more efficient over time, not less.

[i] Rothstein, R. (2011). Fact-Challenged Policy. Policy Memorandum# 182. Economic Policy Institute.

[ii] Carnoy, M., & Rothstein, R. (2013). What do international tests really show about US student performance. Economic Policy Institute, 28.

[iii] Barton, P. E., & Coley, R. J. (2010). The Black-White Achievement Gap: When Progress Stopped. Policy Information Report. Educational Testing Service.

[iv] On these exams, American students have improved substantially, in some cases phenomenally. In general, the improvements have been greatest for African-American students, and among these, for the most disadvantaged. The improvements have been greatest for both Black and White 4th and 8th graders in mathematics. Improvements have been less great but still substantial for Black 4th and 8th graders in reading and for Black 12th graders in both mathematics and reading. Improvements have been modest for Whites in 12th grade mathematics and at all three grade levels in reading (Rothstein, 2011).

[v] Scafidi, B. (2012). The School Staffing Surge: Decades of Employment Growth in America’s Public Schools. Friedman Foundation for Educational Choice.

[vi] Roy, J. (2012). Review of The School Staffing Surge. National Education Policy Center, 3

[vii] As summarized by ANOVA blogger Frederik DeBoer here: https://fredrikdeboer.com/2017/05/16/study-of-the-week-what-actually-helps-poor-students-human-beings/ see also: Dietrichson, J., Bøg, M., Filges, T., & Klint Jørgensen, A. M. (2017). Academic interventions for elementary and middle school students with low socioeconomic status: A systematic review and meta-analysis. Review of Educational Research, 87(2), 243-282.

 

[viii] http://www.uaedreform.org/downloads/2017/04/employer-contributions-for-retirement-4-4-17.pdf

[ix] Teacher Weekly Wage₂₀₁₅                               = $1,092 x 1.20 = $1,311

College Graduate Weekly Wage₂₀₁₅                   = $1,416 x 1.10 = $1,557

Adjusted Share      =   84%

 

Source: The Sweeny-Prieto School Aid Proposal: An Analysis

Mark Weber

Doctoral Candidate

Rutgers, The State University of New Jersey

POLICY BRIEF: Weber_SweeneyPrieto_June26_2017

SUPPLEMENTAL FILE (Regression Output): SweeneyPrietoLog

Executive Summary

This brief presents an analysis of the school funding plan presented by New Jersey Senate President Steve Sweeney and Assembly Speaker Vincent Prieto, referred to here as “Sweeney-Prieto.” The proposal:

• Will drive more aid to districts with higher proportions of Hispanic, free lunch-eligible, and LEP students.
• Will drive less aid toward districts with students classified as having a special education need.
• Will drive more aid on average to districts in the CD District Factor Group; however, there is great variety among these districts, with some losing significant amounts of aid.
• Will give less aid to very small districts.
• Will drive aid towards districts making greater local taxing effort, holding school cost and taxing capacity

While this last characteristic makes Sweeney-Prieto more “fair” overall, there are still individual districts that are receiving significantly less or more aid than would be predicted by measures of cost, capacity, and effort.

In addition, the aid allocated under Sweeney-Prieto is less than 2 percent of the aid proposed by the governor’s budget for FY18; the proposal, therefore, has little overall effect on the bringing New Jersey’s school budgets to adequacy as designated by the state’s own funding law.

Based on these conclusions, I offer the follow recommendations:

• Policymakers should ensure that those districts receiving significantly less aid per pupil under Sweeny-Prieto – particularly those whose changes in aid are far under prediction – do not suffer undue harm from the proposal.
• Lawmakers should carefully consider the unintended consequences of basing the reallocation of aid largely on factors such as the Growth Cap or Adjustment Aid, and adjust the allocation of aid accordingly.
• All stakeholders should realize the scale of Sweeney-Prieto renders it largely ineffective in making up for the chronic underfunding of SFRA over the last eight years.

 

Challenging “dollars belong to the child, not the system” argument. (Tax)$$ provided by society to serve pub interest 1/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Society (Tax $$) provides for the mix of public goods and services that make for a “better” society 2/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Those dollars provided via taxation support a) infrastructure & b) operations for public goods & services 3/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

That includes schools, roads, public safety (policy/fire), national security, public utilities, parks, etc. 4/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Investment in public infrastructure meant to serve not only immediate users, but users for generations to come 5/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Support of operations also not exclusive to those using the service or good today or this year 6/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Contributors of tax dollars include those directly AND indirectly benefiting 7/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Indirect benefits accrued from investment in system of pub schooling include capitalization in housing values 8/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Pub $$ matters not only to those using it here/now. $$ don’t just belong to parents/kids currently school-aged. 9/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Assets acquired with public $$ surely do not belong exclusively to current parents/kids. 10/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

To apply the “money belongs to the child” logic to other public goods/services… 11/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Should we just give public safety tax dollars back to citizens to buy guns & build fences? 12/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

What about fire protection? Buy your own hose & connect it where? “Charter” fire departments instead? 13/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

What about national defense/security? Here’s your voucher – you’re on your own? 14/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Some things we (U.S. Society) decided not to really provide as public good/service–Health Care/Higher Education. 15/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

As inequitable as our K12 system is (it’s bad!), anyone believe higher Ed & health care are actually more equitable? 16/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

Yes,pub spaces give rise to tensions among constituents. Confronting(not avoiding)tensions is net+ for society! 17/ #publicgoodsmatter

— Bruce Baker (@SchlFinance101) December 23, 2016

//platform.twitter.com/widgets.js

 

*Education, or public schooling (public school systems) in particular is not typically considered a “public good” as the provision of public schooling does not comply with the definition of a “pure public good” which can be equally accessed by all, without reduction in benefits to any.  The intent here (in the above tweet-storm) was to shed some light on the importance of understanding the role/position of these publicly financed education systems in society and that there’s more to these systems than the year to year provision of “schooling” to those who happen to be school aged in a specific community at a specific point in time.

Over the past few years, co-authors Preston Green and Joseph Oluwole and I have written a few articles on the use, misuse and abuse of student growth metrics for evaluating teachers and for imposing employment consequences based on those metrics. Neither of our previous articles addressed the use of even more nonsensical status and status change measures for deciding which schools to close or reconstitute (with significant implications for employment consequences, racial disparities, and reshaping the racial makeup of the teacher workforce).

I have written a few posts on this blog in the past regarding school rating systems.

• Ed Waivers, Junk Ratings & Misplaced Blame: Jersey Edition
• Follow up on Ed Waivers, Junk Rating Systems & and Misplaced Blame – New York City’s “Failing” Schools
• Ed Waivers, Junk Rating Systems & Misplaced Blame: Case 1 – New York State

I’ve also tried to explain what might be appropriate and relevant uses of testing/assessment data for informing decision/policymaking:

• Cutting through the Stupid in the Debate over Annual Testing
• Friday Thoughts on Data, Assessment & Informed Decision Making in Schools
• Strolling through the PARCC (data)

It blows my mind, however, that states and local school districts continue to use the most absurdly inappropriate measures to determine which schools stay open, or close, and as a result which school employees are targeted for dismissal/replacement or at the very least disruption and displacement. Policymakers continue to use measures, indicators, matrices, and other total bu!!$#!+ distortions of measures they don’t comprehend, to disproportionately disrupt the schools and lives of low income and minority children, and the disproportionately minority teachers who serve those children. THIS HAS TO STOP!

Preston, Joseph and I previously explained problems of using value-added and growth percentile measures to evaluate and potentially dismiss teachers in cases where those measures led to racially disparate dismissals (under Title VII). We explain that courts would apply a three-part analysis as follows:

Courts apply a three-part, burden-shifting analysis for Title VII disparate impact claims.”-‘ First, the plaintiffs must establish a prima facie case, showing that a challenged practice has an adverse impact on a minority group.”‘* Once the plaintiffs have established a prima facie case, the burden shifts to the employer to show that the employment practice in question has a “manifest relationship to the employment”;”^ in other words, the employer has to show a “business justification.””^ If the employer satisfies this requirement, the burden then shifts to the plaintiffs to establish that less discriminatory alternatives exist.’ ‘^

In other words, showing the disparate effect is the first step. But that doesn’t alone mean that a policy or the measures it relies upon are wrong/unjustifiable/indefensible. That is, if the defendant (state, local district, “employer”) can validate that those measures/policy frameworks have a “manifest relationship” to the employment, then current policy may be acceptable.  Plaintiffs still, though, have opportunity to show that there are less discriminatory alternatives which are at least equally justifiable.

What follows are some preliminary thoughts in which I consider the “usual” measures in state and local school rating policies, and how one might evaluate these measures were they to come under fire in the context of Title VII litigation. It actually seems like a no-brainer, and waste of time to write about this, since few if any of the usual measures and indicators have, from a researchers perspective, “manifest relationship to the employment.” But, the researchers perspective arguably sets too high a bar.

Types of Measures & Indicators in School Accountability Systems: An overview

I begin with a clarification of the distinction between “measures” and “indicators” in the context of elementary and secondary education policy:

• Measures: Measures are based on attributes of a system to which we apply some measurement instrument at a given point in time. Measures aren’t the attributes themselves, but the information we gather from application of our measurement tool to the targeted attributes. For example, we construct pencil and paper, or computerized tests to “measure” achievement or aptitude in areas such as mathematics or language arts, typically involving batches of 50 items/questions/problems covering the intended content or skills. The measures we take can be referred to as “realizations” which are generated by underlying processes (all the stuff going on in school, as well as in the daily lives of the children attending and teachers working in those schools, inclusive of weather conditions, heating, cooling and lighting, home environments, etc.).  Similarly, when we take a child’s temperature we are taking a measure on that child which may inform us whether the child is suffering some illness. But that measure tells us nothing specific of the underlying process – that is, what is causing the child to have (or not have) a fever. If we wrongly assume the measure is the underlying process, the remedy for a high temperature is simply to bathe the child in ice, an unlikely solution to whatever underlying process is actually causing the fever.

• Indicators: Indicators are re-expressions of measures, often aggregating, simplifying or combining measures to make them understandable or more “useful” for interpreting, diagnosing or evaluating systems – that is, making inferences regarding what may be wrong (or right) regarding underlying system processes. Indicators are best used as “screening” tools, useful for informing how we might distribute follow-up diagnostic effort.  That is, the indicators can’t tell us what’s wrong, or if anything really is wrong with underlying processes, but may provide us with direction as to which processes require additional observation.

Measures are only useful to the extent that they measure what we intend them to measure and that we use those measures appropriately based on their design and intent.  Indicators are only useful to the extent that they appropriately re-express and or combine measures, and do not, for example, result in substantial information loss or distortion which may compromise their validity or reliability. One can too easily take an otherwise informative and useful measure, and make it meaningless through inappropriate simplification.

Expanding on the body temperature example, we might want to develop an indicator of the health of a group of 100 schoolchildren. Following typical school indicator construction, we might simplify temporal body temperature readings for a group of 100 children to a binary classification of over or under 98.6 degrees. Doing so, however, would convert otherwise potentially meaningful (continuously scaled) data into something significantly less meaningful (if not outright junk). First, applying this precise “cut-score” to the temperature ignores the margin of error in the measurement, establishing a seemingly substantive difference between a temperature of 98.6 and 98.7, where such a small difference in reading might result from imprecision of the measurement instrument itself, or our use of it. Second, applying this cut-score ignores that a temperature of 103 is substantively different from a temperature of 98.7 (more so than a difference between 98.6 & 98.7).  Given the imprecision of measurement (where temperature measurement is generally more precise than standardized testing), if large shares of the actual temperatures lie between 98.6 and 98.7 degrees, then large numbers will likely be misclassified. The over/under classification scheme has resulted in substantial information loss, limiting our ability to diagnose issues/problems with underlying processes. We’ve taken an otherwise useful indicator, and converted it into meaningless junk.

Notes on Validity and Reliability

As noted above, for a measure to be useful it must measure what we intend it to measure, and we must be using/interpreting that measure based on what it actually measures. That is, the measure should be valid, which takes the forms of “face validity” and “predictive validity” (there are many additional distinctions, but I will limit the discussion herein to these two).  A test of “algebraic reasoning” should measure a student’s capacity to apply algebraic reasoning to test items which accurately represent the content of “algebraic reasoning.” That is, content validity, which relates to face validity.

“Predictive validity” addresses whether the measure in question is “predictive” of a related, important outcome. This is particularly important in K-12 education systems where it is understood that successful test-taking is not the end-game for students. Rather, we hope these assessments will be predictive of some later life outcome, starting, for example, with higher levels of education attainment (high school graduation, college completion) and ultimately becoming a productive member of and/or contributor to society.

Measures commonly used for evaluating students, schools and education systems can actually have predictive validity without face validity. Typically, how well students perform on tests of language arts is a reasonable predictor (highly correlated with) of how well they also do on tests of mathematics. But that doesn’t mean we can or should use tests of language arts as measures of mathematics achievement. The measures tend to be highly correlated because they each largely reflect cumulative differences in student backgrounds.

The measures should also be reliable. That is, they should consistently measure the same thing – though they might consistently measure the wrong thing (reliably invalid). If measures are neither reliable nor valid, indicators constructed from the measures are unlikely to be reliable or valid. But, it’s also possible that measures are reliable and/or valid, but indicators constructed from those measures are neither.

Often, conversion of measures to indicators compromises either or both face or predictive validity. Sometimes it’s as simple as choosing a measure that measures one thing (validly) and expressing it as an indicator to measure something else, like taking a test score which measures a students’ algebraic reasoning ability at a point in time, and using it as an indicator of the quality of a school, or effectiveness of that child’s teacher.

Other times, steps applied to convert measures to indicators, such as taking continuous scaled test scores and lumping them into categories, can convert a measure which had some predictive validity into an indicator which has little or none. For example, while high school mathematics test scores are somewhat predictive of success in college credit bearing math courses, simply being over or under a “passing” cut-score may have little relation to later success in college math, in part, because students on either side of that “passing” threshold do not have meaningfully different mathematics knowledge or skill.[1]

Aggregating a simplified metric (to proportions of a population over or under a given threshold) may compound the information loss.  That is, by looking at the percent of children over or under and arbitrary and likely meaningless precise bright-line cut-score through an imprecise though potentially meaningful measure provides little useful information about either the individuals or the group of students (no less the institution they attend, or individuals employed by that institution).

Misattribution, Misapplication & Misinterpretation

Far too often, face validity is substantially compromised in indicator construction.  The problem is that the end users of the indicators often assume they mean something they simple don’t and never could.  A common form of this problem is misattribution – or asserting that measure or derived indicator provides insights into an underlying process – where in fact, the measures chosen, their re-expression and aggregation provide little or no insight into that process.  I, along with colleagues Preston Green and Joseph Oluwole explain the misapplication of student growth measures in the context of teacher evaluation. Student growth indicators (Student Growth Percentiles) are rescaled estimates of the relative (to peers) change in student performance (in reading and math) from one point in time to another. They do not, by their creators’ admission, attempt to isolate the contribution of the teacher or school to that growth. That is, they are not designed to attribute growth to teacher or school effectiveness and thus lack “face validity” for this purpose.[2] But many state teacher evaluations wrongly use these indicators for this purpose. In the same article, and second related article[3], professors Green, Oluwole and I explain how related approaches, like Value-added modeling at least attempt to isolate classroom or school correlates of growth, partially addressing face validity concerns, but still failing to achieve sufficient statistical validity or reliability.

Neither of our articles addresses the use of crude, school aggregate indicators, constructed with inappropriately reduced versions of assessment measures, to infer institutional (school) or individual (teacher) influence (effectiveness), leading to employment consequence.  That is because these indicators clearly lack the most basic face validity for making such inferences or attribution, leading to employment consequence.  As such, we felt it unnecessary to bother critiquing these indicators for this purpose. As noted above, proportions of children over/under arbitrary thresholds on assessments tell us little about the achievement of those children. These indicators are aggregations of meaningless distinctions made through otherwise potentially meaningful measures. These aggregations of meaningless distinctions surely provide no useful information about the institutions (and by extension the employees of those institutions) attended by the students on which the measures were originally taken.

In the best case, the un-corrupted measure – the appropriately scaled test score itself – may be an indicator reflecting ALL of the underlying processes that have brought the child to this point in time in their mathematics or language arts achievement, knowledge or skills. Those processes include all that went on from maternal health through early childhood interactions, community and household conditions, general health and well-being along the way (& related environmental hazards), and even when entering school, the greater share of hours per day spent outside of the schooling environment.  Point in time academic achievement measures pick up cumulative effects of all of these processes and conditions, which are vastly disparate across children and their neighborhoods, which is precisely why these measures continue to reveal vast disparities by race and income, and by extension, across schools and the often highly segregated neighborhoods they serve.[4]

The Usual Indicators, What they Mean & Don’t Mean

Here, I provide an overview of the types of indicators often used in state school report cards and large district school rating systems.  Simply because they are often used does not make them valid or reliable. Nor does it provide the excuse for using these indicators inappropriately – such as misattribution to underlying processes – when the inappropriateness of such application is well understood.  Table 1 provides a summary of common indicators.  The majority of indicators in Table 1 are constructed from measures derived from standardized achievement tests, usually in math and English language arts, but increasingly in science and other subject areas.  State and local school rating systems also tend to include indicators of graduation and attendance rates.

Of all of the indicators listed in Table 1, only one – Value-Added estimates – attempts attribution of “effect” to schools and teachers, though, with questionable and varied success. Most (except for the most through value-added models) are well understood to reflect both socioeconomic and racial bias, at the individual level and in group level aggregation. More detailed discussion of these indicators follows Table 1.

Table 1. Conceptual Overview of School Performance Indicators

	Indicator Type	Facial	Notes	Attribution / Effect
Academic assessment (e.g. reading/ math standardized test)	Scale score or group mean scale score	Student or group status/ performance level	All such measures norm referenced to an extent, even if attached to supposed criteria (content frameworks)	Makes NO attempt to isolate influence of schools or  teachers [no manifest relationship]
	Percent “proficient” or higher (or above/below any status cut-point)	Status of a group of students relative to an arbitrary “cut-score” through distribution	Ignores that those just above/below threshold not substantively different. Substantially reduces information (precision)	Makes NO attempt to isolate influence of schools or  teachers [no manifest relationship]
	Cohort Trend/ Change	Difference in status of groups sequentially passing through a system	Typically measures whether subsequent group has higher share over/under threshold than previous. Influenced by differences in group makeup, and/or differences in test administration from one year to next.	Makes NO attempt to isolate influence of schools or  teachers [no manifest relationship]
	Growth Percentiles	Change in student test score from time=t to time=t+1	Usually involves rescaling data based on student position in distribution of student scores. Does not account for differences in student background, school or home context (resources)	Makes NO attempt to isolate influence of schools or  teachers [no manifest relationship]
	Value-Added	Change in student test score from time=t to time=t+1 conditioned on student and school characteristics	Uses regression models to attempt to compare growth of otherwise similar students in otherwise similar settings. Ability isolate classroom/school “effects” highly dependent on comprehensiveness, precision & accuracy of covariates.	Attempts to isolate relationship between influence gains and classroom factors (teachers) and schools. Suspect in terms of manifest relationship.[5]
Persistence & Completion	Graduation Rates / On-Time Progress / Dropout Rates	Student Status / Performance Level	Tracks student pathways through grade levels, courses against expectations (on track)	Makes NO attempt to isolate influence of schools (resources, etc.) or  teachers [no manifest relationship]
Attendance	Proportion of “enrolled” students “attending” per day, averaged over specified time period	Status of a group of students relative to an arbitrary “cut-score” through distribution	Typically does not discriminate between types/causes of absences. Known to be disparate by race/SES, in relation to chronic health conditions.[6]	Makes NO attempt to isolate influence of schools (resources, etc.) or  teachers [no manifest relationship]

Common indicators constructed with standardized assessment measures are summarized below:

• “Proficiency” Shares: Shares of children scoring above/blow assigned cut-scores on standardized assessments. Few states or districts have conducted thorough (if any) statistical analysis of the predictive validity of the assigned cut-points or underlying assessments.[7] Raw scores underlying these indicators capture primarily cumulative differences in the starting points and backgrounds of students, individually and collectively in schools and classrooms. Proportions of children over/under thresholds depend on where those arbitrary thresholds are set.
  • Whether raw scores or proficiency shares, these indicators are well understood to substantially (if not primarily) reflect racial and socio-economic disparity across students and schools.
• Change in “Proficiency” Shares (Cross-Cohort Percent Over/Under): For example, comparing the proficiency rate of this year’s 4^th grade class to last year’s 4^th grade class in the same school, perhaps calculating a trend over multiple cohorts/years. These indicators capture primarily a) changes in the starting points and backgrounds of incoming cohorts of students (demographic drift), and b) changes in the measures underlying the accountability system (test item familiarity, item difficulty, etc.), whether by design or not. In many cases, year over year changes in shares over/under proficiency cut-scores are little more than noise (assuming no substantive changes to tests themselves, or cohort demography) created by students shifting over/under arbitrary cut-scores with no substantive difference in their achievement level, knowledge or skill.
  • These indicators do not tend to reflect racial or socio-economic disparity (except for trends in those disparities) in large part because these indicators usually reflect nothing of substance or importance, and are often simply noise (junk).[8]
• Student Achievement Growth (Student Growth Percentiles): Constructed by comparing test score growth of each student, with respect to others starting at similar points in the distribution, among their peers. These indicators include only prior scores and do not include other attributes of students, their peers or schooling context. These indicators capture differences in student measures from one point in time to another, and are most often (read always, in practice) scaled relative to other students, so as to indicate how a students’ “growth” compares with an average student’s growth. These indicators may also capture primarily the conditions under which the student is learning, both at home and in school.
  • These indicators tend to be racially and socio-economically disparate because they control only for differences in students’ initial scores, and not for students school and peer context, or students’ own socio-economic attributes.[9]
• Value-Added Model Estimates: Based on statistical modeling of student test scores, given their prior scores, and various attributes of the students, their peers and schooling context (breadth of factors included varies widely, as does the precision with which these factors are measured). These measures are the “best attempt” (as I often say “least bad” alternative) to isolate the school and/or classroom related factors associated with differences in student measures from one point in time to another, but cannot, for example differentiate between school resources including instructional materials, building heating, cooling and lighting and/or the “effectiveness” of employees.[10]
  • These indicators may substantively reduce racial and economic disparity of the measures on which they are based, by using rich data to compare growth of similar students in similar contexts.

This is not to suggest that value-added measures in particular have no practical use. But rather that they should be used appropriately, given their limitations. As Preston Green, Joseph Oluwole and I explain:

Arguably, a more reasonable and efficient use of these quantifiable metrics in human resource management might be to use them as a knowingly noisy pre-screening tool to identify where problems might exist across hundreds of classrooms in a large district. Value-added estimates might serve as a first step toward planning which classrooms to observe more frequently. Under such a model, when observations are completed, one might decide that the initial signal provided by the value-added estimate was simply wrong. One might also find that it produced useful insights regarding a teacher’s (or group of teachers’) effectiveness at helping students develop certain tested skills.

School leaders or leadership teams should clearly have the authority to make the case that a teacher is ineffective and that the teacher even if tenured should be dismissed on that basis. It may also be the case that the evidence would actually include data on student outcomes – growth, etc. The key, in our view, is that the leaders making the decision – indicated by their presentation of the evidence – would show that they have reasonably used information to make an informed management decision. Their reasonable interpretation of relevant information would constitute due process, as would their attempts to guide the teacher’s improvement on measures over which the teacher actually had control. (p. 19)[11]

Other measures receiving less attention (Why “attendance” rates are sucky indicators too!)

School rating reports also often include indicators of student attendance and attainment and/or progress toward goals including graduation. It must be understood that even these indicators are subject to a variety of external influences, making “attribution” complicated. For example, one might wrongly assume that attendance rates reflect the efforts of schools to get kids to attend. If schools do a good job, kids attend and if they don’t, kids skip.  If this is the case, there should be little difference in attendance rates between “rich schools” and “poor schools” unless there is actually different effort on the part of school staff.   This assertion ignores the well understood reality that children from lower income and minority backgrounds are far more likely to suffer from chronic illness including asthma, obesity or both, which is a strong predictor of chronic absence (>10).[12] Additionally, neighborhood safety affects daily attendance.[13] These forces combine to result in significant racial and economic disparities in school attendance rates which are beyond the control of local school personnel.

Once again… for my thoughts on productive use of relevant measures in education, see:

• Cutting through the Stupid in the Debate over Annual Testing
• Friday Thoughts on Data, Assessment & Informed Decision Making in Schools

Notes:

[1] See for example, http://usny.nysed.gov/scoring_changes/MemotoDavidSteinerJuly1.pdf and/or Papay, J. P., Murnane, R. J., & Willett, J. B. (2010). The consequences of high school exit examinations for low-performing urban students: Evidence from Massachusetts. Educational Evaluation and Policy Analysis, 32(1), 5-23.

[2] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71.

[3] Green III, P. C., Baker, B. D., & Oluwole, J. (2012). Legal and Policy Implications of Value-Added Teacher Assessment Policies, The. BYU Educ. & LJ, 1.

[4] Duncan, G. J., & Murnane, R. J. (Eds.). (2011). Whither opportunity?: Rising inequality, schools, and children’s life chances. Russell Sage Foundation.

Coley, R. J., & Baker, B. (2013). Poverty and education: Finding the way forward. Educational Testing Service Center for Research on Human Capital and Education.

Reardon, S. F., & Robinson, J. P. (2008). Patterns and trends in racial/ethnic and socioeconomic academic achievement gaps. Handbook of research in education finance and policy, 497-516.

[5] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71.

Green III, P. C., Baker, B. D., & Oluwole, J. (2012). Legal and Policy Implications of Value-Added Teacher Assessment Policies, The. BYU Educ. & LJ, 1.

[6] http://www.changelabsolutions.org/sites/default/files/School-Financing_StatePolicymakers_FINAL_09302014.pdf

[7] Some concordance analyses relating ISAT scores to ACT “college ready” benchmarks have been produced. See: http://www.k12accountability.org/resources/Early-Intervention/Early-intervention-targets.pdf & http://evanstonroundtable.com/ftp/P.Zavitkovsky.2010.ISAT.chart.pdf

[8] See, for example: http://www.shankerinstitute.org/blog/if-your-evidence-changes-proficiency-rates-you-probably-dont-have-much-evidence

[9] See also: Ehlert, M., Koedel, C., Parsons, E., & Podgursky, M. (2014). Choosing the right growth measure. Education Next, 14(2) & https://njedpolicy.wordpress.com/2014/06/02/research-note-on-teacher-effect-vs-other-stuff-in-new-jerseys-growth-percentiles/ & http://www.shankerinstitute.org/blog/does-it-matter-how-we-measure-schools-test-based-performance

[10] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71.

Green III, P. C., Baker, B. D., & Oluwole, J. (2012). Legal and Policy Implications of Value-Added Teacher Assessment Policies, The. BYU Educ. & LJ, 1

[11] Baker, B. D., Oluwole, J., & Green, P. C. (2013). The legal consequences of mandating high stakes decisions based on low quality information: Teacher evaluation in the race-to-the-top era. Education Evaluation and Policy Analysis Archives, 21, 1-71. http://epaa.asu.edu/ojs/article/view/1298/1043

[12] http://www.changelabsolutions.org/sites/default/files/School-Financing_StatePolicymakers_FINAL_09302014.pdf

[13] Sharkey, P., Schwartz, A. E., Ellen, I. G., & Lacoe, J. (2014). High stakes in the classroom, high stakes on the street: The effects of community violence on students’ standardized test performance. Sociological Science, 1, 199-220.

About a year ago, I released this report: http://nepc.colorado.edu/publication/charter-revenue

In which Gary Miron and I discuss various methods by which charter school operators work largely within existing policy constraints, to achieve financial gain. While working on this report, I explored various other topics that did not make the final cut, in part because I was then, and continue to have difficulty gathering sufficient information. The other day, however this article was posted on LA Weekly about wage extraction by “Gulen” charter schools: http://www.laweekly.com/news/did-a-california-charter-school-group-fund-an-effort-to-overthrow-the-turkish-government-7666698

This reminded me that I still had the related content I’ve posted below sitting on my hard drive, and that I should at least get around to posting on the blog.

Accessing Money by Creating a “Company Store”: Taxing Salaries through Affiliated Enterprises

There are many ways to “tax” teacher wages, and recapture a share of wages through closely affiliated entities (see Figure 6). In recent years, for example, there has been significant reporting on charter schools using imported labor for classroom teaching.[i] This staffing model provides two opportunities. First, there exists an opportunity to engage generally in non-traditional compensation agreements with imported labor, which may include much lower and differently structured salaries and benefits than would be paid to traditional domestic, certified teaching applicants. Second, there exist additional opportunities to “tax” the wages of these employees for such services as processing their visas and/or making travel arrangements. These services may be provided by private entities closely affiliated with the schools. That is, the money flows from one hand to the other. In this case, that money is obtained by obligating employees to pay a tax from their wages.

Public districts have been involved in similar schemes. In 2012, Louisiana school districts were caught up in a scheme involving Filipino teachers forced into exploitive contracts through a Los Angeles based placement firm.[ii] No evidence was presented, however, of any kick-back relationship between the placement firm and the districts. The arrangement came to light because (lawyers on behalf of) teachers brought litigation against the placement firm.

It can be difficult if not impossible to obtain documents specifying financial parameters of agreements between foreign teachers and charter school operators, because charter school operators refuse to release these documents, claiming exemption from public disclosure laws. Blog sites including Charterschoolwatchdog.com has posted partial documents indicating that Turkish teachers in U.S. charter schools teach under a “Tuzuk” (roughly translated as regulations or bylaw) involving an agreement to receive only a nominal stipend for their work; and, they are required to remit cash portions of their salaries as well as retirement benefits and tax refunds (although it is difficult to discern exactly which entity receives remittances).[iii]

Figures 7 and 8 illustrate salary structures derived from available statewide personnel files in Houston, Texas and the state of New Jersey. Figure 7 shows that Harmony/Cosmos schools in Texas, which rely heavily on Turkish labor,[iv] show relatively flat distribution of salaries by experience levels. It may be that actual salaries allocated show this distribution or that schools have simply reported a single salary figure of $40,000 for all teachers regardless of position, education or experience level, and not representing actual compensation. Other charter operators in Houston show growth in salaries by experience parallel to that of Houston district schools. KIPP schools pay higher than district salaries.

Figure 7. EMO Operated Charter vs. Houston ISD: Full Time Teacher Salaries by Experience at Constant Degree Level in 2010

Figure 8 shows that New Jersey schools similarly reliant on Turkish labor (Paterson Science and Technology, Central Jersey College Prep, among others[v]) also report flat salaries with respect to experience. Again, by contrast, the KIPP school (TEAM Academy) a) pays higher than other charter schools and public districts and b) shows growth with respect to experience. So too does North Star Academy, and Uncommon School. Yet, the schools employing larger shares of Turkish labor show a) very low salaries, b) little or no growth with respect to experience and c) in some cases, no variation at all in salaries with respect to experience.

Data Source: salaries modeled using statewide “Fall Staffing Report” data. Salaries estimated as a function of degree level and years of in-state experience, excluding administrative job codes, and using data from 2010 to 2015. Predictions based on teachers holding a bachelors degree in 2015.

Figure 8. New Jersey Charter School Salary Structure (by Total Experience)

Other options exist for recapturing portions of teacher salaries. But as is true for the Turkish Tuzuk, documentation of these schemes may be difficult to obtain from privately-managed charter schools that often claim these agreements are exempt from public disclosure laws. One common model is the “company store,” where employees are required to purchase goods and/or services from the affiliated entities. This model can be used for visa processing fees for foreign labor, but might also be used for obtaining relevant credentials, professional development, or even housing.[vi]

For example, founders of New York and Newark, NJ area charter schools and management companies have established their own Graduate School of Education (Relay GSE), staffed primarily by themselves—current and former employees of the charter schools and management companies. Relay GSE was criticized in public hearings over its use of under-credentialed and inexperienced faculty to deliver its programs, but was eventually granted accreditation.[vii]

The Relay Board of Trustees includes founders of KIPP, NYC; Achievement First; and affiliates of Uncommon Schools.[viii] In New Jersey, Relay’s graduate programs are offered on-site within North Star Academy,[ix] a Newark charter school affiliated with the Uncommon Schools network (established by a founder of Relay GSE). The Dean of Relay, Newark, is a co-founder of North Star Academy.[x] Former teachers from the affiliated charter schools report being obligated as a condition of employment to obtain credentials (MA degrees and related certifications) from Relay GSE. That is: employees at the charter schools are having a portion of their salary taxed to pay tuition to a “graduate school” run by founders of their own charter schools, operated within their own charter school facility (lease agreement unknown), where courses are often taught by their own teaching peers having only slightly more advanced education and experience.[xi] We elaborate on this example in Appendix A.

Another way for affiliated charter schools to channel money to Relay is to set aside a portion of their budget to subsidize graduate education—but only at Relay GSE. That is, some EMOs (including Uncommon) have a practice of paying for graduate degrees obtained from Relay, but not from any other institution (unless the teacher can prove that Relay does not offer a degree in the same field). Teachers agreeing to pursue their degrees from Relay with school support must complete those degrees or, as noted earlier, are required to reimburse their EMO for any/all tuition reimbursement they received.

Compensation taxing models could conceivably be taken one step further, to access not only a portion of teacher salaries, but also their retirement benefits, by adopting the Enron version of the Company Store: having employee retirement benefits invested directly back into the charter school operator or through a closely affiliated financial manager. In the Enron case, approximately 60% of the company’s 401(k) was invested in Enron stock.[xii]

The information provided in the main text describes a “company store” model at Uncommon Schools/Relay Graduate School of Education. But it only describes the information that is publicly accessible. A great deal more about the model is still unclear. Here we provide a few insights, based on the limited disclosures from, e.g., former employees. We stress here that it is difficult to obtain clear official documentation of the practices involved, because current employees are sworn to secrecy, including prohibitions against providing contractual documents and employee handbook materials. Accordingly, this appendix summarizes merely what we were able to ascertain through published accounts and acquired documents. This is an obvious area for officials and investigative journalists to delve into, since a great deal is unknown, and the limited accounts, discussed here, have yet to be fully corroborated.

A former Uncommon schools teacher who identified herself only as “Emily” created a blog with a single post, which states: “upon my being hired and deciding to take the job in August 2011, I was required to enroll at Relay. It was one of the conditions of my employment—to begin the process of getting my masters in education at Relay, which has a partnership with Uncommon Schools.”[xiii] That is, as we note above, obtaining the graduate credential was a condition of continued employment.

Regarding financial arrangements, the former Uncommon Schools teacher notes that “Relay offers all of its students a very doable tuition package—you can pay in small increments that are realistic with most charter school salaries. And, to be a Relay student, you must be employed at a charter or public school. Not only that, Relay will greatly subsidize your tuition from various sources that fund it, such as money from AmeriCorps.” This squares with Relay’s own explanation of “Tuition & Aid” from their web site:

Figure A1

 

 It also squares with language we acquired from an Uncommon Schools employee handbook:

Figure A2. Handbook Excerpt

These limited pieces of information, taken together, suggest that if total annual tuition for Relay graduate programs is $17,500, the typical Uncommon Schools new teacher can be expected to pay about half that for two years (to degree completion) and have the other half subsidized. If the teacher leaves her position at Uncommon Schools prior to completion or otherwise fails to complete the graduate degree, the teacher may be responsible for reimbursing the school. New teachers would seem to have little choice in the matter.

Additional evidence gathered from individual school websites and through deep web searches (leading to Uncommon Schools Family Handbooks) reveals school faculty bios that indicate large shares of teachers who have a) recently completed degrees at Relay, b) are currently pursuing degrees at Relay, and/or c) currently serve as adjunct instructors for Relay.

Again, these elements of the Company Store need further investigation. If they are true, it raises serious concerns. If it is not, then taxpayers and public officials should expect public clarification on that point.

As we discussed in the main text of this brief, a small handful of individuals were involved in the founding and original governance of each institution. Our analysis here is not comprehensive, because Uncommon Schools and Relay GSE span many state boundaries. Thus our examples here focus primarily on the New York/New Jersey area, with particular attention to Newark, NJ. Table A1 shows the board membership and highest paid employees for four interconnected organizations (and there may be others we have missed thus far).

The two left hand columns include Uncommon Schools (data from IRS 990, 2014, 2013 tax year) and North Star Academy’s Consolidated Annual Financial Report (CAFR) from the New Jersey Department of Education. We expect these organizations to overlap because one (North Star) is subsidiary to the other (Uncommon). The right three columns include Relay Graduate School of Education; a separate non-profit entity providing financial support to Relay, called Uncommon Knowledge and Achievement, Inc. (UK&A); and a third organization, Zearn. We also expect these three organizations to overlap because it would appear that UK&A’s primary function is to raise funds for providing scholarship/aid for students attending Relay GSE (providing $500,000 to GSE in tax year 2013 according to IRS filings, and another $100,000 to Zearn). We understand less about the role of Zearn, except that it appears to be a company created for the aggregation and distribution of electronic teaching materials/modules for mathematics.

More significant potential concerns arise when parties cross over between the left two and right three columns, especially in significant governance or leadership roles or where significant financial interests are involved. Specifically, it would be questionable for leadership of Uncommon Schools to be requiring their employees to purchase goods/services from Relay if in fact those leaders have financial interests in Relay. This is likely true even if these individuals absolve themselves of voting roles in setting those policies. IRS Filings (and charter school financial audits) indicate that Uncommon Schools is a “related party” to its various subsidiary schools (and even their real estate arms). IRS filings similarly acknowledge the relationship between Relay, UK&A and Zearn. But, for some reason neither IRS filings nor external audits of Uncommon Schools mention a formal relationship between Uncommon and Relay, despite Norman Atkins’ role as CEO of one and chair of the board of the other.

The connectedness between Relay and Uncommon does not end there. Jamey Verrilli is Dean of Relay’s Newark, NJ branch, which operates out of an address shared by a North Star (Uncommon) school (10 Washington Place, Newark). Verrilli also sits as a “community member” representative on the board for North Star. That is, he sits on the board of a school that obligates (as we understand it) its teachers to pay his other organization, housed in the same building, substantial annual tuition. In addition, Relay Dean Verrilli’s wife is employed by North Star (duly noted on state conflict of interest reports).[xiv]

Deeper exploration of the Uncommon and Relay websites reveals that Uncommon’s Newark Managing Director also plays a significant role in Relay GSE, leading Relay’s Leverage Leadership program. Other Uncommon employees, including those in leadership roles, also have significant presence on the Relay website (noted in green in Table A1), including the starring role of Verrilli’s wife in demonstration videos.[xv] In sum, formal leaders of Uncommon/North Star Academy are in positions to influence policy decisions that would direct school resources to Relay and would require employees to make payments to Relay, as well as senior employees and administrators at North Star in a position to exert significant coercion over incoming and other junior staff. Note that according to the handbook, these individuals wield significant influence over what types of professional development and graduate pursuits might be approved for reimbursement.

Table A1

	CMO and Subsidiary School (Acknowledged as Related)		UK&A Formally Related to Relay GSE (Acknowledged)
	Uncommon Schools (IRS 990)	North Star Academy (NJ CAFR)	Relay GSE (IRS 990)	Uncommon Knowledge & Achievement Inc. (IRS 990)	Zearn LLC (IRS 990)
Board Members
	Norman Atkins (Chair)	Rick Rieder (Chair)	Larry Robbins (Chair)	Norman Atkins (CEO)	Norman Atkins (Chair)
	David Cooper	Michael Lytle	David Levin (Secretary)[1]	Timothy Saintsing (COO, Prior)	Evan Rudall (CEO)
	Donald Katz	Robert Howitt	Arthur Levine (Trustee)	Thackston Lundy (COO, Current)	Shalinee Sharma (Secretary)
	Charles Harris	Gia Rys	Julie Mikuta (Trustee)		Robert Karr (Treasurer)
	Pearl Kane	Nicole Albano	David Saltzman (Treasurer)		David Levin
	Robert Karr	Paul Bambrick-Santoyo	Dacia Toll (Vice Chair)[2]		David Saltzman
	Rondo Moses	Trisha Scipio-Derrick	Carlos Watson (Trustee)		Dirk Ziff
	Robert Jain	Ravi Bellur			Dacia Toll
	Brooke Reid	Scott Sleyster			Larry Robbins
	Neal Moszkowski	Nkyah Taylor
Employees & Officers
	Brett Peiser (CEO)	Carolyn Hack (Treasurer)	Norman Atkins (President)
	Carolyn Hack (CFO/COO)	Michael Ambriz (COO)	M Yvonne Chao (CFO)
	Paul Bambrick-Santoyo (Managing Director, Newark)	Julie Jackson (Principal)	Piper Evans (Director of Finance)
	Julie Kennedy (Managing Director)	Mike Mann (Head of School)	Timothy Saintsing (COO)
	Joshua Phillips (Managing Director)	James Verrilli (Community Member)	Mayme Hostetter (Dean, Relay NY)
	Julie Jackson (Managing Director)		Michael Larosa (CIO)
	Dana Lehman (Managing Director)		Brent Maddin (Provost)
	Barbara Martinez (Chief External Officer)		James Verrilli (Dean, Relay Newark)
	Laura Lee McGovern (Chief of Staff)		Robert Underwood (CTO)
	Laura Maestas (Chief Talent Officer)
	Tara Marlovits (COO Uncommon NYC)
	Michael Ambriz (COO North Star Academy)
Overlapping Others with current/past significant presence on Relay web site [see, for example: http://www.relay.edu/programs/leverage-leadership-institute-fellowship-0/overview] [1] KIPP Co-founder [2] Founder, Achievement First

Figure A3 attempts to sketch out some of these relationships, with particular attention to Newark, NJ. On the left are charter schools, where not only Uncommon Schools, but also KIPP and Achievement First are official partners of Relay GSE. Clearly, as indicated in the table above, there exists significant, substantive leadership and financial overlap between Uncommon and Relay. In addition to the founding/leadership overlap, in Newark the two share a facility (financial arrangement unknown) and seem also to share at least a few employed staff, including the Newark Managing Director (compensated at $223,227 by Uncommon alone in tax year 2013).

About 50% of Relay tuition is directly extracted from new faculty, assuming the above sources to be correct, with the other 50% coming through: a) schools like North Star and b) UK&A.

Figure A3

One might question how lucrative it can be to rely heavily on tuition extraction (and other subsidies) from only new, incoming teachers required to pursue two-year graduate programs. After all, how many new teachers might there be in any given charter school? Could there possibly be a constant or increasing flow? Figure A4 uses statewide annual staffing files from the New Jersey Department of Education to tally the total numbers of teachers and numbers of teachers with zero or one-year experience in North Star Academy from 2009 to 2015. In the most recent three years, North Star alone had 103, 134 and then 153 novice teachers. Assuming that Relay could gross $17,500 per year for each, the total would approach $6 million by 2015 (about half extracted from teacher wages and half from grants made by or through related entities). The high rates of novice teachers in North Star and other Uncommon (NY) schools are a function of both growth and very high rates of turnover.

Figure A4

A Closing Thought

So, I know many will say, what’s the big deal? Relay affiliated charter schools are (at least perceived as) highly successful and their success reliant on strict adherence to their specific pedagogy. So why not grow their own/train their own teachers? As I see it, that’s a clearly separable issue from what I outline above. First of all, having effective staff in-service training, even where provided by affiliates is more acceptable in a number of ways (depending on how those affiliate contracts are handled & disclosed). It’s localized, specific training, not degree/credential granting having broader public policy implications. No institution – a single public or private school  – should be granted the authority to self-credential their own employees in this way, to hold supposedly broad, transferable expertise, which is in fact anything but.  Provide specific training for their current job and context? OK. Provide degrees and credentials w/broader state accreditation? That’s the public policy concern (which should have been shut down earlier).  The second (ethical/COI) concern (which is also a public policy concern) is that no school leaders should have authority to require or have such opportunity to coerce teachers under their supervision to buy back services provided by their employer or close affiliates of their employer (resulting in financial benefit).

I am quite sure that if I was to collaborate with my Dean at Rutgers Grad School of Education to start a charter charter school chain in and around New Brunswick, placing us on the leadership team and board (along with other faculty and leadership at Rutgers),

AND THEN, require each of our new teacher employees in those charter schools to obtain graduate degrees from Rutgers GSE as a condition of maintaining their employment (even if indirectly, that we needed to give them assignments which required they be certified in certain areas, in which we provide the cert programs),

AND ON TOP OF THAT, use teachers who had just completed those programs to deliver the courses (as underpaid graduate assistants) to the new teacher on site… in the charter schools…

it would raise at least a few eyebrows.

And it should!

Additional Documents

relaygradsched-certauth-signed-sec-7-27-16

And a musical tribute:

NOTES

[i]  Pilcher, J. (2014). Charter schools use Turkish ties, visas to get teachers. Cincinnati.com http://www. cincinnati.com/story/news/2014/10/05/charter-school-turns-turkish-teachers/16791669/

Beauchamp, S. (2014). 120 American Charter Schools and One Secretive Turkish Cleric. The Atlantic. http:// www.theatlantic.com/education/archive/2014/08/120-american-charter-schools-and-one-secretive-turkish- cleric/375923/

Saul, S. (2011). Charter Schools Tied to Turkey Grow in Texas. New York Times. http://www.nytimes. com/2011/06/07/education/07charter.html?_r=1

[ii] Samuels, D. (2012). Filipino teachers win $4.5 million jury verdict, claim they were forced into ‘exploitative contracts.’ The Times-Picayune. http://www.nola.com/news/baton-rouge/index.ssf/2012/12/filipino_ teachers_win_45_milli.html

[iii] See, for example, the blog: http://www.charterschoolwatchdog.com/tuzuk—a-contract-to-steal.html

[iv] Saul, S. (2011). Charter Schools Tied to Turkey Grow in Texas. New York Times. http://www.nytimes. com/2011/06/07/education/07charter.html?_r=0

[v] Schools selected based on information provided here: http://turkishinvitations.weebly.com/list-of-us-schools. html

[vi] See for example: http://www.teachers-village.com/index.html

[vii] Mooney, J. (2013). Alternative Grad School Raises Concerns About Who’s Teaching NJ’s Teachers. NJ Spotlight. http://www.njspotlight.com/stories/13/07/09/alternative-grad-school-raises-concerns-about-who- s-teaching-nj-s-teachers/

[viii] See for example: http://www.relay.edu/about/people

[ix] See for example: http://www.relay.edu/campuses/newark (it is unknown whether Relay pays rent for use of space at North Star Academy).

[x] See for example: http://www.relay.edu/campuses/newark

[xi] See, for example: http://emily-thetruth.blogspot.com/2012/03/truth-about-relay-graduate-school-of. html?spref=tw

[xii] Schutz, E.E., Francis, T. (2002). Enron Executives’ Benefits Kept on Growing as Retirement Plans of

Employees Were Cut. Wall Street Journal. http://www.wsj.com/articles/SB1011745748757428600

[xiii] Blog post can be found here: http://emily-thetruth.blogspot.com/2012/03/truth-about-relay-graduate-school- of.html

[xiv] State ethics filings here:

http://education.state.nj.us/ethics/discforms.php?c=13;d=7320;s=100330;dl=20140201

http://education.state.nj.us/ethics/discforms.php?c=13;d=7320;s=134341;dl=20150201

[xv] To understand the extent of connections between Relay/Uncommon, see here (https://www.youtube.com/ watch?v=9XnrT4MC7fw) where Beth Verrilli (employed by North Star), wife of Jamey Verrilli (Dean, Relay Newark) is used in a demonstration video, linked from Relay GSE leadership program, but to an Uncommon Schools web site. So, in the video, Mrs. Verrilli is acting as an employee of Uncommon Schools (North Star), while being used in a demonstration for leadership training at Relay, applying techniques promoted by Bambrick Santoyo – Managing Director of Uncommon Newark, but also lead faculty member for Relay’s Leverage Leadership program.

I haven’t revisited the Newark charter school data in a while now. Here are my most recent cuts at the most recent available data:

First, DEMOGRAPHICS of District and Charter Schools

RED bars indicate CHARTER schools. Blue are other Newark Schools. North Star highlighted w/diagonal pattern as it has been the subject of recent twitter/blog fodder/debate.

[reason for focusing on “free lunch” explained here]

Next, COHORT reduction rates:

NJ Cohort Analysis Stata Code

Given the reshuffling of students across district regular and special schools, including reorganization during the period, the following is a much clearer comparison than the ones previously posted (aggregating “district” schools):

How, you ask, can a senior cohort exceed in size the 7th graders who started five years prior? Presumably, some of the many, many black boys who started in North Star in particular (but also TEAM) do end up back in NPS district schools.

& Finally, some modeled estimates of relative GROWTH and relative PARCC Scale Scores (relative to predicted values) – noting that I a unable to effectively account for high rates of attrition (as in the North Star case specifically).

 

 

New Report: http://www.shankerinstitute.org/resource/deconstructing-myth-american-public-schooling-inefficiency

Here’s the summary:

In this paper, we begin by classifying the arguments that assert American schools are relatively inefficient into two categories: the long-term trend argument and the international comparison argument. Our focus herein is on the latter of these two. We then describe two frameworks for approaching either of these arguments: cost efficiency and production efficiency. We explain that the typical spending/outcome model used to make the case that the United States is a relatively inefficient nation is wholly unsuitable for drawing these or any conclusions. Accounting for differences in student populations is helpful, but still inadequate for building a model that can be used to assess a country’s relative efficiency. Evaluating education inputs such as teacher wages and class sizes can further refine comparisons between nations; however, it is unlikely that even these refinements are enough to conduct analyses that can credibly back claims about the relative efficiency of America’s education system. That said, an appropriately limited analysis can still inform our understanding of how the U.S public education system compares with systems in other countries.

What does this all mean?

• First and foremost, we can say with some confidence that existing expositions of U.S. inefficiency (based on Organization for Economic Cooperation and Development [OECD] national spending data and Program for International Student Assessment [PISA] scores) are so lacking in methodological rigor that they are of little if any value in public discourse or for informing national education policies.
• Second, it is unlikely that we could ever obtain data of sufficient precision, accuracy and comparability to meet the demands of more legitimate efficiency modeling for cross-national, intercontinental analyses.

Any and all comparisons using OECD and related data should be conducted with consideration of the limitations discussed herein. But some insights might be drawn from our analyses:

• Among other things, the OECD per-pupil spending measure, as incomparable as it is, shows that the U.S. may have higher per-pupil spending than many nations, but falls right in line with expectations for nations of similar gross domestic product (GDP) per capita.
• The U.S. is both a high-spending and high-GDP country, but some of that high education spending may be a function of the scope of services and expenses included under the education umbrella in the U.S.
• We also know that despite seemingly high spending levels in the United States, teachers’ wages lag with respect to other professions, and the wage lag is not a result of providing relatively smaller class sizes.
• In fact, our primary class sizes (roughly equivalent to schooling provided from about age 5 through 11 or 12 years of age) are average and lower secondary (roughly equivalent to schooling provided from about 12 to 16 years of age)[1] class sizes large. Our wage lag is, to an extent, a function of high non-teaching wages (related to our high GDP per capita), necessarily making it more expensive to recruit and retain a high-quality teacher workforce.

To summarize: The U.S. is faced with a combination of seemingly high education expense, but noncompetitive compensation for its teachers, average to large class sizes, and a high rate of child poverty. Again, it’s hard to conceive how such a combination would render the U.S. comparable in raw test scores to low-poverty nations like Korea or Finland, or small, segregated, homogeneous enclaves like Singapore or Shanghai.[2]

Finally, it is equally important to understand the magnitude and heterogeneity of the U.S. education system in the context of OECD comparisons, which mainly involve more centralized and much smaller education systems. Lower-poverty, higher-spending states that have been included in international comparisons, like Connecticut and Massachusetts, do quite well, while lower-spending higher-poverty states like Florida do not. This unsurprising finding, however, also tells us little about relative efficiency, and provides little policy guidance for how we might make Florida more like Massachusetts, other than by waving a wand and making it richer, more educated and perhaps several degrees colder.

[1] See UNESCO (2012). Note that the 2011 ISCED classification scheme does not specify age ranges, focusing instead on the purposes of the levels of education (basic preparation, etc.). But that system is crosswalked to the 1997 scheme which does specific age ranges and where the previous (1997) ISCED level 1 and level 2 remain aligned with the present (2011).

[2] Shanghai in particular has several mitigating factors that make comparing its scores to other nations highly suspect; see: http://www.brookings.edu/research/papers/2013/10/09-pisa-china-problem-loveless