Why something like this is difficult to analyze without context

I was reading an article in US News and World Report called “How States are Spending their Money in Education.” This is the type of article that I see every once in a while that is worth discussing. The article is linked here.

I am not criticizing the author here, more so I am providing a warning for readers not to fall into a snap judgements with this. The article shows per pupil expenditures and can lead readers (depending on viewpoints about schooling) into a couple of different camps. On one hand, this type of article allows for readers to point to the numbers and say “State X is spending too much money” while “State Y is not spending enough money.” A different argument would be that since “State X is spending more than State Y, State X must give a better education.” Both arguments are tempting, but beware, both are not sophisticated enough to be informative.

The first reason financial comparisons are difficult is because cost of living varies in each state. For example, one would expect the cost of living in New York City to cause everything in the surrounding communities to be more expensive. This, in turn, is probably going to make schooling more expensive. Teachers need more money to live, transportation is more expensive to run, and building operation costs are higher. Judging quality, or even “bang for your buck” is difficult with just dollar figures.

Furthermore, just because a state spends more money, it does not mean students will perform better in school. There are a lot of factors that go into the production of great students (pardon the crude expression) so simply looking at funding will not tell the reader the whole picture. Often lower-income students require much more money than higher incomes students to get better academic results for a variety of reasons, so states with higher spending may appear to get less of a “bang for their buck” if they have needier students who require more services. Thus running a simple statistical test  such as a correlation with per pupil expenditures and student test scores (which I did, for a 2013 national achievement proxy) would probably not leave you with strong results because there are so many factors in the schooling picture. When I ran a simple correlation I got a positive, but very low correlation between state per pupil expenditures and student scores — which based on the explanations of my post here, one would expect.

So where am I going with all of this?

I certainly believe money in education is EXTREMELY important. Needy students and higher cost of living certainly will inflate costs for achieving great schools. However, understanding local contexts and needs is essential and simply looking at per pupil expenditures across states does not provide researchers with much meaningful information. It is a start, but we need to have more than this to have the right information to make judgements about efficacy and quality. Metrics like purchase power (instead of dollar-based per pupil expenditures) and statistical controls for student-level and community-level factors are  other pieces of vital information. So if you are reading this article and you are in Washington DC with a high per pupil expenditure, I am here to say, well, yeah that should be the case because you live in a very expensive city with a needy population. Your per pupil expenditure have to be higher to provide the services your students need. In fact, it might not even be high enough.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s