I stumbled across this interesting post about education at The Simple Dollar titled “Why Johnny Can’t Read: Simpson’s Paradox and the Greatly Exaggerated Death of American Public Education.” It reports on results of a study comparing outcomes of public & private school education titled “Comparing Private Schools and Public Schools Using Hierarchical Linear Modeling” [pdf]. The critical part of the post is that the initial difference in performance showing an advantage for private schools:

Right off the bat in this report, when discussing fourth grade reading, the report says “the average private school mean reading score was 14.7 points higher than the average public school reading score.” Wow, that’s a talking point right there, isn’t it? From just that sentence alone, media reports condemning the public schools in America flew forth.

disappears once other factors are controlled for:

But the next sentence is quite interesting: “After adjusting for selected student characteristics, the difference in means was near zero and not significant.” What does that mean? It merely means if they divide the fourth graders up into smaller pieces of pie (such as divisions based on race, income level, and so forth) and compared equivalent pie pieces between private schools and public schools, there was no statistical difference between private schools and public schools.

The blog report goes on to conclude:

Private school test scores are better because the demographics of their students are tilted highly towards groups that do well no matter whether they’re schooled in public or private schools.

If you’re paying for private school, you’re not paying for a great education – you’re paying for demographics. The report actually says that on average, your student won’t do better on standardized tests in private school compared to public school; the only thing that makes a difference is their race, their income level, and their parents’ involvement in their education.

In other words, position matters. In this case, the private/public school difference masked other significant differences (income, race/ethnicity, etc.).

They go on to report that international comparisons between U.S. students and those in other countries are often similarly flawed because again there are factors within the general category (U.S. vs. other nation) that hide underlying causes to educational performance differences. For example, they report that in South Korea:

only 60% of their students of high school age actually attend high school; the “bottom” 40% are actually funneled into separate vocational schools. Thus, when standardized tests compare 12th graders in Korea and 12th graders in the United States, if you individual compare demographic groups, the United States does at least as well as Korea, but if you combine all students as a whole, South Korea appears to dominate the United States. That’s because their high schools, much like private schools in the United States, are full of students whose personal demographics are universally geared towards greater educational success regardless of the schools.