New study of universal preschool has confusing results...

I've been thinking a lot this past week about reading, vocabulary, and poverty, but I haven't gotten my thoughts in order. In the meantime, I want to note an interesting recent study of universal preschool that raises some interesting questions about what school is good for. The pandemic has taught me that school is important in all sorts of ways, and this study goes along with that lesson.

Preschool leads to more schooling, but test scores don't go up

The new paper, which came out last month, is about the long-term effects of a universal preschool program in Boston. The study finds that preschool enrollment significant boosts high-school graduation, SAT-taking, college attendance, and reduces juvenile incarceration. Interestingly, the study finds that preschool enrollment does NOT have a detectable impact on state achievement test scores. This is an interesting paradox: universal preschool keeps kids in school longer and gets kids to college, but it does not measurably increase their test scores. How could this be?

This finding--that increased attachment to school does not have a detectable impact on test scores--is extremely counterintuitive. School is supposed to make you better at reading and math, and if preschool makes you more likely to stay in school, more likely to go to college, more likely to take the SATs, more likely to stay out of trouble, those behaviors should lead you also to get more of schools benefits, and you should do better on reading and math tests. That this apparent increase in attachment to school does not lead to higher scores seems to suggest, disturbingly, that school itself does not necessarily lead to higher scores. This uncomfortable suggestion reminds me a paper from a couple of years ago about the "summer slide."

The summer slide is famous, but it might not really exist

The "summer slide" refers to the idea that students--especially less privileged students--not only fail to make academic progress over the summer, but actually slide backward. This idea has long been accepted as an established truth. Malcolm Gladwell's book, Outliers, states the idea this way:

"America doesn't have a school problem. It has a summer vacation problem." (Outliers, 260)

The idea is that the gap between rich kids and poor kids narrows during the school year, and then widens during the summer. According to a 2019 paper (von Hippel and Hamrock), this isn't actually true: 

"If summer learning gaps are present, most of them are small and hard to discern."

The 2019 paper is an interesting reminder of how much measurement matters. Learning gaps were thought to widen over the summer not only because testing regimes used different tests for different grades, and scores on the different tests were hard to compare, but also because how you measure the gap affects how much a gap changes. Scaling test results in different ways results in dramatically different accounts of how much a gap widens or narrows:

Ultimately, von Hippel and Hamrock's paper suggests that "reading and math gaps grow substantially more in the first five years of life than they do in the nine years after school begins." 

If the gap is established in early childhood, shouldn't preschool help?

Von Hippel and Hamrock's 2019 paper concludes with a call for investment in early childhood: 

"The growing interest in early-childhood programs—such as preschool, home visits, and new-parent training—is justified. It is vital to invest in early-childhood programs, and it is just as vital to understand why some early-childhood programs succeed in shrinking gaps, whereas others fail to realize their potential."

This makes sense. But, to come back to the paper about preschool, if preschool can help, why didn't universal preschool in Boston raise test scores?

What is the point of school, after all?

Right now, we don't know why the test scores didn't go up. But one deep question this study raises is: What is the point of school?, is the purpose of school to raise test scores, or is it to help students attain other skills or qualities (persistence? obedience? cooperation? organization?). Is it maybe okay that test scores didn't go up? I don't really like that idea, since I think academic skills are fun and useful, but I am open to the idea that school has different kinds of benefits as well, and maybe those are more important.

Deus Ex Scholia (or, Michael Lewis has a bad idea)

Post-pandemic, I'm coming back to this blog as a place to think through questions about education and literacy. Because education matters, but it's not always obvious how it matters, nor how it can be improved.

Deus Ex Scholia

It's always amazing how many fairly sophisticated people have naive and simple-minded ideas about school. I listened not too long ago to an interview, about America's pandemic response, with the writer Michael Lewis. It was mostly smart and interesting, but there was a moment that stopped me short.

Lewis and the host were talking about whether or not different policy responses to COVID actually made much difference, given that states like Florida, whose policies were quite lax, didn't seem to have much higher rates of transmission. Culture is, it seems, at least as important as policy. But what can you do to change the culture? This is an interesting question, so I started listening more carefully.

Talking about policies that might actually make a difference, Lewis said, "I’ll be God for a minute and institute reforms that sound kind of technical, but I think would have massive consequences." The first two things Lewis would do as God could conceivably actually make a difference: first, turn many leadership positions in the Federal government from political appointments to civil service positions with a longer tenure, and second, reward the little people who are doing the hard work. 

So far, so good: those things wouldn't be easy to accomplish, but they are interesting and slightly unusual ideas. But when Lewis offered his third suggestion, I laughed aloud. I'll quote the paragraph in full:

Number three, I reintroduce civics into the American curriculum. So every kid needs to know — you can’t get out of the eighth grade without knowing what the Department of Energy does and things like that. We start to educate again the population about what the public sector is. There’s such a branding problem right now. There’s such a screwed up notion about what it’s for and what it’s not for.

This ought to be self-evidently absurd, but unfortunately it's the kind of thing you hear a lot from people who want to change the culture.

When little children or simpletons think a cultural phenomenon is dangerous, they say, "There ought to be a law."

At a somewhat higher level of sophistication, people realize that maybe a "law" is a crude form of coercion, or they think that a law won't be passed unless people's beliefs change first, so they say, "Let's teach about this in school." As if this would be easy, as if you could make sure that the teaching on the subject that ended up in school wouldn't be the exact opposite of what you want, and as if, even if you could teach what you wanted, the teaching would be effective. I'm skeptical about each of these assumptions (and I'm not alone)!

Is school material or cultural? (Yes.)

School is important, but it's not always important in a simple, straightforward way, and it can't always be changed in a simple, straightforward way. We want kids to get better at reading, so we spend a lot of time "teaching" them to read. When this doesn't work as well as we want it to, we critique the way we're teaching them, but not whether direct instruction is the best way for kids to learn. When kids from different socioeconomic backgrounds perform dramatically differently on tests, we decide that it is school's job to fix this difference and we introduce new layers of programming, or we de-track our classes, or we change the curriculum. When this doesn't work as well as we want it to, we critique the way we are teaching, but not whether school can actually be expected to compensate for large inequities outside the schoolhouse.

The deus ex scholia method of solving social problems in the world is as absurd and unsatisfying as deus ex machina method of solving plot problems in a narrative. In fact, it's much less satisfying, since the deus ex scholia method just puts off the solution into the future.

I tend to see cultural problems as downstream of material problems. If we want to solve cultural problems, let's make sure people have good-paying jobs, reliable health insurance, and housing that doesn't cost a fortune (and smaller-bore material changes like the ones Michael Lewis mentioned could also be helpful).

Does that mean that schools aren't important? No, of course not. Schools can make a difference, but in schools, too, the material is as important as the cultural. Direct instruction in literacy or rhetorical techniques is useful, in small doses, but most of what students need is actual practice in reading and writing and speaking. Don't ask, what is the teacher saying; ask, what is the student actually doing?

I'll be thinking about these questions more in future posts. For now, it's summer, and I'm going for a bike ride.