What do the available data tell us about NYC charter school teachers & their jobs?

This post is about rolling out some of the left over data I have from my various endeavors this summer.  These data include data from New York State personnel master files (PMFs) linked to New York City public schools and charter schools, NYC teacher value-added scores, and various bits of data on New York City charter and district schools including school site budget/annual financial report information.

Here, I use these data combined with some of my previous stuff, to take a first, cursory shot at characterizing the teaching workforce of charter school teachers in New York City. All findings use data from 2008 to 2010.

To summarize the following figures, New York City charter school teachers:

1. Are relatively inexperienced (but not all in their first 3 years)
2. Are young (but not all 22 or 23 years old)
3. Have longer contract years
4. Are paid well for their experience, with a portion of the additional pay covering the additional time
5. Have smaller classes to teach
6. Work in schools that spend much more than surrounding district schools
7. Work in schools that serve much less needy student populations than surrounding district schools
8. Have 4^th grade students with relatively “average” to below average scale score outcomes compared to schools serving similar populations
9. In some cases, have 8^th grade students with high average scale score outcomes compared to schools serving similar populations
10. Where data were available, have value-added scores which vary from the citywide average in both directions, with KIPP being the lowest and Uncommon schools the highest (in the aggregate). Notably, Uncommon schools also have consistently smaller class sizes and the fewest low income students.

That’s it. Nothing particularly surprising. Nothing astounding. No miracles, but, a subset of schools that are in some ways different from district schools. Further, they are different in ways that perhaps aren’t that sexy… aren’t that “reformy.” Salaries rise with experience and they just happen to rise faster than district school salaries in some cases. Class sizes are small… especially at the middle school level… even though we are often told that class size reduction is only important in early grades. And, well, the outcomes are kind of mixed.

Experience & Age

The first figure shows that the typical NYC charter school teacher has about 6 tears experience in 2010. Some anomalies occur in the 2008 data. NYC district school teachers have about double that. But, it should be noted that some of this difference is likely explained by the average age of the schools themselves, in addition to higher turnover (a topic for a later date).

One would expect that as the schools mature, their staff will also somewhat – unless they actually make a concerted effort to excess teachers beyond a specific experience level. If they do begin to accumulate more experienced staff, they will also accumulate the higher expenses associated a more experienced staff and accumulated retirees (assuming any stay long enough to retire from these schools).

The average charter school teacher is between 25 and 30 years old, compared to the average district school teacher being around 40.

I’ll admit, I’m having some trouble reconciling average years of experience being around 6 and age around 25, but it would appear for the most part that the experience and age line up such that most went directly into teaching from their undergraduate studies. That is, there aren’t likely many later career changers here.

Contracts & Compensation

Personnel master file data report that teachers in Harlem Village Academies, Achievement First, KIPP, Uncommon and Lighthouse schools are on 11 and 12 month contracts. Harlem Childrens’ Zone was similar but did not report 2010 data. By contrast BOE contracts were 10 month (according to averages taken from full time individual teachers in the data).

Average salaries were highest in 2010 in KIPP schools, with BOE schools second, and other high profile charter chains not far behind. However, these averages are a function of degree levels, experience levels and contract months.

So, we can use a regression model to isolate those effects and compare “otherwise similar” teachers, and to determine the wage differential associated with the additional contract months (to the extent that contract months vary within the charters).

This next figure uses a regression model to compare the salaries of “otherwise similar” teachers by job classification, full time status, degree and experience level.  On average, a teacher with all of the same attributes can make a salary that is $8,000 higher in an Ucommon school and nearly $6,000 higher in an Achievement First School. Teachers in HCZ and KIPP schools make an annual salary about $4,000 higher than those in BOE schools.  Controlling for months worked, KIPP and HCZ salaries are comparable to district salaries and Uncommon and Achievement First salaries remain higher.

I’ve often asserted that many teachers might be willing to/interested in working a longer year for a higher annual salary. After all, it might be desirable to earn more doing what you do best and are professionally trained to do, rather than searching for other part time Summer work that does not always take advantage of your expertise.  That may not be the case for all teachers, but would likely be the case for at least some (as it is for these charter teachers). More pay for more time. Makes sense.

The following figure uses the regression model to lay out the predicted salaries for a teacher in 2009 across experience levels. Notably, much like BOE school salaries, NYC charter teacher salaries do increase with experience. In particular, consistent with what I’ve shown previously for salaries in Texas, New Jersey and Connecticut high profile charters, Achievement First and Uncommon schools pay what appears to be a significant salary premium. None pay significantly less than district schools, and none are flat with respect to experience (like Gulen school salaries in NJ or TX). KIPP and others track for the most part with district salaries.

Resources & Working Conditions

Now, not to beat an issue into the ground, but the following figure summarizes the per pupil operating expenditure differences between NYC charters and district schools (with the data becoming cleaner, more accurate and precise each time I take a swipe at them). These margins of difference are ever so slightly higher than those in my previous study, but the model is similar. High profile charters in NYC substantially outspend district schools serving similar student populations.

I’ve addressed critiques of these figures previously and have provided sensitivity analysis and discussion here: https://schoolfinance101.wordpress.com/2012/05/07/no-excuses-really-another-look-at-our-nepc-charter-spending-figures/

The full length report on these and related figures is here:

• Baker, B.D., Libby, K., & Wiley, K. (2012). Spending by the Major Charter Management Organizations: Comparing charter school and local public district financial resources in New York, Ohio, and Texas. Boulder, CO: National Education Policy Center. Retrieved [date] from http://nepc.colorado.edu/publication/spending-major-charter.

Now, for this next figure, I use a regression model to compare the demographics of NYC charter schools to district schools, where the dependent measure is the population characteristic, and the independent measures include a) grade range served, b) year of data and c) borough of location of the school. That is, these comparisons are of the student population compared to same grade level schools in the same borough.

Uncommon schools in particular have uncommonly low rates of the lowest income children. This is typical in New Jersey as well (for North Star Academy in Newark). They also have low LEP/ELL rates, but so do they all. Achievement First and KIPP schools also have low rates of the lowest income children – quite a bit lower than district schools. Success Academies in particular have very low rates of LEP/ELL children. All have relatively low rates of children with disabilities.

These differences – when combined – make for very different school environments than for district schools in the same borough.

Now, despite serving much less needy student populations, these charter schools tend to have much smaller class sizes than district schools serving the same grade levels. HCZ schools have small elementary and middle school class sizes – from six to eight fewer students in a class than district schools. Uncommon schools have very small middle grades classes, as do Village Academies and Achievement First Schools. Interestingly, despite their financial dominance, KIPP schools have smaller class sizes than district schools, but not to the extent of others (note that some of KIPPs financial resources are allocated to such endeavors as KIPP to College).

A really interesting twist here is that the emphasis on smaller classes appears to be at the middle school level.  My gut instinct as a former middle school science teacher tells me that this makes sense. But, if one sticks strictly to the strongest findings in research literature, one would expect targeting at the elementary level and would have less justification for targeting small classes to the middle level. But perhaps small middle school class sizes are actually part  of the secret sauce of successful charter schools?

NYSED School Report Cards: https://reportcards.nysed.gov/

Outcomes

Now, with smaller class sizes, higher paid teachers working longer years, and less needy students we would not only expect a higher average performance level, but we’d also likely expect some higher average rates of gain in these schools.  And lottery based studies of NYC charters have revealed some positive findings of differences in performance between lotteried in and lotteried out students. It’s critically important to understand that while the sorting process may have been randomized in these studies the contexts – peer groups, etc. – into which they’ve been sorted is anything but. Again, different peers, different levels of resources – class sizes – length of year – teacher salaries, etc.  And some of this stuff is the stuff of difference that is of particular interest for setting policy.

The figures I offer below are merely descriptive. Again I use regression models to compare the outcomes of schools that are similar in terms of students served. But regression models in this case are used for descriptive purposes (as they often are). I’m really just describing the average difference in outcomes between the charters and schools serving similar populations. In the third slide below, I do use the average teacher value-added outcomes for those schools – but note that very few teachers in any given NYC school actually have sufficient numbers of students in tested grade levels for generating the value added estimates.

In the first figure here, we see that 4th grade assessment performance in many NYC charters is, on average, lower than district schools. The vertical axis indicates the number of points (around a mean near 680) above or below the mean of similar district schools. Most differences are well less than 1 standard deviation ( which is about 17 pts., but a standard deviation would be a pretty big difference. Again, these are levels, not gains.). 4th grade math scores are higher in Uncommon and Achievement First Schools. Most other 4th grade scores are similar to or lower than district schools. In v1 (version 1) models, I use % free or reduced price lunch (which varies little across many of these schools) and in Version 2, I use percent Free only. Charter schools compare less favorably when I evaluate them on the basis of their free lunch populations alone.

Uncommon Schools and Village Academies appear somewhat stronger on their eight grade performance, while others are a mixed bag. Again, these are level – scale – scores, relative to schools serving similar populations. Now, there has been at least some discussion of attrition as a factor in Village Academy performance. I’ve personally found attrition to be a major issue in Uncommon’s Newark School, North Star. But that’s an issue for another day.Matt Di       Carlo over at Shankerblog did a nice explanatory piece on the role of attrition the other day.

Here, what we have are average scale scores that are quite a bit higher than “demographically similar” schools. That said, when I account for free lunch instead of free or reduced lunch, those differences are somewhat muted. That is, when I compare against schools with comparable shares of the lowest income children, positive charter performance differences are muted and negative ones larger.

Again, Harlem Chidlrens’ Zone scores are relatively low, especially when comparing on the basis of fee lunch (v2 models). KIPP scores are pretty much comparable to demographically similar schools and Achievement First and others a mixed bag.

Finally, this figure shows the average value-added scores for teachers in these schools. Uncommon Schools are the only ones that appear to have noticeably above average teachers. One might stretch the data here beyond their capacity to argue that perhaps Uncommon schools are getting results for the premium they pay their teachers… and the fact that they generally provide those teachers with smaller classes. Not an entirely unreasonable story, though a) the underlying dynamic is likely far more complex than this and b) the value added metrics used here are anything but stable and/or decisive.

NYC Teacher Value Added Scores: http://www.ny1.com/content/top_stories/156599/now-available–2007-2010-nyc-teacher-performance-data#doereports

Implications

So, what does this all tell us? Perhaps not much… while at the same time, quite a bit more than we may have already know. At the very least, these data should serve to clear up common misconceptions & reformy misrepresentations of the NYC charter sector.

No… these schools are not staffed by peace core like, minimum wage missionaries (not that I’ve really heard this one all that much). They’re getting paid reasonably and getting paid a premium for at least some of the extra time and effort. Is it enough to sustain the model? Can these schools afford this down the road? Who knows?

No… these schools are NOT proving that money and class size don’t matter.  That would be difficult to prove with a subset of schools that for the most part spend more than district schools and provide smaller class sizes.

And No… these schools are not totally and invariably kicking butt on all student outcome measures, be they performance level measures or performance gains.

And finally… No… these schools are not doing it all… with the same kids!

But this isn’t just a laundry list of stuff to try to hold against these schools. Rather, it’s an attempt to lay out more clearly with publicly available data, what’s going on here, in an effort to move the conversation forward, beyond the usual talking points and into stuff that may really matter!