What might the new NAEP scores mean?

The "Nation's Report Card," otherwise known as the report on the most recent National Assessment of Educational Progress, or NAEP (I always thought it rhymed with "Jeep," but it may rhyme with "tape"), has just come out.  The NAEP is widely considered a good test, and since the NAEP is not high-stakes, there is little reason for schools or students to cheat, or for schools to attempt to teach directly to it.  I remember Mike Dukakis telling an auditorium full of Leafstrewn students that NCLB was unnecessary because we already had the NAEP.

The NAEP is a good test, and because it is such a good test, and because reading and math abilities change so extraordinarily slowly, there aren't the kinds of variations in the NAEP that there are in, say, the MCAS.  MCAS scores have gone up dramatically over the past fifteen years, while the NAEP scores are always more or less the same.  Therefore, the release of the NAEP scores is a little like a Rorschach  test.

Optimists who favor ed reform will point to little fluctuations and say, See, NAEP scores have risen over the past four years!  People who are against ed reform will say, Look, 17-year olds reading scores are below where they were in the nineties! Nerds will rightly point out that the demographics of the students taking the test have changed significantly over the years, and that breaking out subgroups can be interesting (for instance, black kids' scores have improved much more than those of white kids).  Contrarians might say, the scores haven't changed much, so school doesn't matter.

I'd like to point out an interesting feature of the new report and suggest a possible explanation for it.  I'm not sure I'm right, but my explanation goes along with some of what I have said in the past about the difference between short-term thinking and long-term thinking.

NAEP scores rise for 9 and 13 year-olds
The new "Nation's Report Card" has a very clear lede, and here it is:


Both 9- and 13-year-olds scored higher in reading and mathematics in 2012 than students their age in the early 1970s . Scores were 8 to 25 points higher in 2012 than in the first assessment year. Seventeen-year-olds, however, did not show similar gains. Average reading and mathematics scores in 2012 for 17-year-olds were not significantly different from scores in the first assessment year.


In other words, the most significant result coming out of this year's data is that over the past forty years, scores of nine and thirteen-year olds have gone up somewhat, but scores of seventeen-year olds are not significantly different than they were forty years ago.

One way to interpret this is to say, Well, we sure are doing a better job in the elementary schools; but high schools just aren't getting better.

This view--that elementary schools are improving but high schools are not--doesn't seem totally unreasonable, but there's an interesting problem with it that I have not seen anyone point out. The problem is that this result doesn't fit well with the standard theories of education, which would expect increased achievement by K-8 students to lead directly to increased achievement by 17-year-olds. The standard theories of education--that is, the views of people like Tim Shanahan, the ed reformers, and so on--see learning as essentially a step-by-step process of learning skills. According to this view, it is important to teach reading early, and to focus heavily on skills.  But if you have this view, then the higher scores of 9 and 13 year-olds would predict higher scores of 17-year-olds, and so the lower scores of 17-year-olds would seem to imply, not only that high schools were not getting any better, but that they were actually getting significantly worse.  For you would expect that if you took two 13-year-olds and put them through the same secondary education, the one who was a better reader as a 13-year-old would end up a better reader at 17. For a much better reader at 13 to end up the same as his less-skilled peer at 17 would seem to imply a much worse secondary education.

Have US high schools gotten worse over the past 40 years?
The NAEP scores are, if you follow the standard model, evidence that high schools have actually gotten worse over the past forty years.  This is certainly possible. It's also possible that there are demographic issues involved (changes in dropout rates could affect the scores of 17-year-olds).  But there is another possibility: it could be that the short-term thinking that has been increasingly prevalent over the past few decades has actually led to short-term success, but to a kind of short-term success that has not supported long-term improvement.

I can see two ways this could work.  One is a direct cost: the short-term teaching could be actively bad in the long run.  For instance, it might turn kids off to learning or reading.  If you drill kids for tests, the drilling might improve their scores but make them less creative thinkers.  Two, there might be an opportunity cost: by teaching skills or teaching to the test, you might do less of the kinds of things that prepare kids for learning later on.  For instance, you might read aloud to the kids less, or you might cut down on recess, or you might reduce the time allotted for free, creative play.  Any of these could be imagined to result in lower reading scores a decade later.

Evidence that this kind of short-term/long-term trade-off might be possible can be found in the studies on Waldorf schools that I wrote about a couple of weeks ago.  Waldorf schools do not do any explicit teaching of reading skills until the second grade, and their reading scores are, not surprisingly, markedly below those of other schools in the early grades:



The Waldorf schools do, however, catch up.  I'd love to see data on where those students are at the age of 17.  It seems very possible that by the age of 17 the Waldorf kids, who while everyone else was drilling on phonics instead did a lot of listening to stories, singing songs, reciting poems, and observing nature, might be way ahead.  And that may be the same dynamic we see in the NAEP scores.


If our goals are long-term, why are we all thinking short-term?
As I have repeatedly argued, getting better at reading is a very long-term process, and yet far too much of our thinking as teachers, like far too much of the discourse about education, focuses on the short term.  Most discussion of lesson-planning, for instance, seems based on teaching students a discrete skill that they have never attempted before.  John Hattie, in the introduction to his magnum opus, Visible Learning, offers my favorite example of this short-term thinking: Hattie describes in loving detail the excitement of an initial lesson in rappelling down a building, and then says that this is "the heart of the model of successful teaching and learning."  This is absurd, since reading is neither dangerous nor novel to most of our students, and Hattie would find that teaching rappelling to people who'd already been rappelling for ten years would be very, very different. But Hattie's absurdity is just an extreme version of the kind of thinking we all do.  We are after all called "teachers," and just as it is natural to think to ourselves, "So, what particular skill am I going to teach today?", it is also natural to want to measure students' improvement over the relatively short time periods of a unit, a semester, or a year.  So, under MCAS and NCLB, we now give students high-stakes tests every year, and we are moving to a system by which teachers are evaluated by the results of these short-term assessments.

I doubt this is wise, but in the end it may not do too much harm. The remarkable stability of the NAEP scores is a healthy reminder that changes in educational regimes in the US have not made much difference to test scores. On the other hand, the NAEP may also mean that short-term thinking is ineffective: it may lead to short-term successes, but those short-term successes do not necessarily lead to longer-term success.

The Talk of the Town: Malcolm Gladwell, non-fiction, thrillers, and the delusions of realism

(A piece online in the NYTimes has been getting a lot of attention in the past few days.  Here's a quick response.)

Perhaps our students do need to read a bit more non-fiction
The Common Core calls for having kids read more non-fiction.  A recent post on the NY Times website says that reading non-fiction is a good idea because it will help students write non-fiction.  This is the best argument I've seen for reading non-fiction, and it echoes something I've said for years: that it is absurd to ask students to write analytical essays when they never read them.  Nevertheless, the most important thing is that kids read a lot. If you read a lot, it’s easier to learn to write.  And if you can write, you can write anything, more or less.  You just have to look at some examples first.

When I taught Leafstrewn's Senior creative writing course (those were the days!), I structured the whole course around literary apprenticeships.  As a class, we read, studied, and then emulated a number of authors; in the culminating project, students read, studied, and emulated an author of their own choosing.  The course was easily the most satisfying and coherent teaching I've ever done.

For most kids, creative writing comes more easily than analytical writing.  This is probably partly because most kids have read a fair amount of creative writing but very, very little analytical writing.  If, therefore, we want our kids to get better at writing analytical essays, we should have them read some analytical essays—but not necessarily thousands of pages of them.

Any reading that’s not pleasure reading should have a clear short-term purpose
The Times post starts off with a very compelling story about Malcolm Gladwell reading a hundred Talk of the Town pieces before writing one of his own.  But the post draws the bizarre conclusion that this anecdote supports the Common Core's drastic (and, like the rest of the Common Core, totally non-data-driven) call for fully half of students' reading to be of "informational texts."

This conclusion is silly.  First of all, the argument suffers from one of the great weaknesses of educational discourse, the tendency to overly short-term thinking.  Reading 100 Talk of the Town pieces probably took Malcolm Gladwell about two hours.  Those two hours were no doubt excellent short-term preparation for the immediate task before him, but surely no one would argue that those two hours were what made Gladwell a successful New Yorker writer.  Gladwell's writerly skill was probably the result of decades of reading, and if we want our students to be able to write more like Malcolm Gladwell, the question we should be asking is not, What did Malcolm Gladwell read for the two hours before he wrote his first Talk of the Town piece, but what did he read for the decades before that?

Fortunately, my amazing research skills were able to provide some insight into Gladwell's reading habits.  The first sentence of the first search result of the first thing I typed (typoed, actually) into the google box on my browser ("lamcolm gladwell childhood reading") provided this testimony from the man himself:

"I am, first and foremost, a fan of thrillers and airport literature, which means the number of books that I read this year that reach the literary level of the typical New Yorker reader is small."

Now, I love Jack Reacher just as much as the next guy, but I would not use Gladwell’s reading list to argue that the best preparation for writing for the New Yorker is to read a lot of Lee Child novels.  Instead, our students should, over the long term, be reading a lot of whatever interests them, and then, in the short term, be reading more targeted texts for specific purposes.

Pleasure Reading
The Gladwell anecdote highlights not only the distinction between short-term and long-term thinking, but another important distinction as well: the difference between pleasure reading and purposeful reading.

As I've said before, I think of pleasure reading not as skimming, but as a deep immersion of the kind that can remove you from yourself and your surroundings.  Gladwell, like many travelers, likes thrillers for precisely this escapist reason (it's less clear why people would want to bring them to the beach!).  For me, as I get older, non-fiction increasingly provides this deep immersion, but in any case the purpose is largely the pleasure, the immersion--and the benefits of this immersion are comfort and facility with language.  These benefits accrue only over the long term.  A course of pleasure reading will not show many practical results in a time frame of less than a year or so (though I have learned a thing or two from Jack Reacher about guns and Ford Crown Victorias). Pleasure reading also strikes me as different from "informational text," which is the Common Core term for non-fiction.  The NYTimes post used The Immortal Life of Henrietta Lacks as an example of the non-fiction that students should be reading. That book would serve well--but for pleasure reading, not for a specific purpose.

Purposeful Reading
Purposeful reading is pretty different from pleasure reading.  This activity is almost always of short-term benefit. For students, there are two main purposes: (1) to get information; (2) to learn how to write a particular kind of text.  The first purpose is more important for Social Studies, Science, and other "content area" courses, and the second is more important for English class.  Since I think reading is a very effective way to learn, I think the main place for purposeful reading of informational text is in the content areas, and the main purpose should be to get information.  Ideally, these informational texts should also be pleasurable--I never learned much history in school, and most of what I know about history comes from adult pleasure reading, mostly of the popular kind—but in a social studies course the purpose is primary, and it is, again, short-term.  Deep knowledge and understanding will no doubt accrue over the long-term, but the short-term purpose is informational.
The second kind of purposeful reading—that aimed at craft, at learning how to write a particular kind of text—is the main reason for English class ever to privilege non-fiction.  The excellent journalism program at Leafstrewn has students read regularly, but the reading is always purposeful, aimed at the immediate goal of improving the students own writing—learning how to write a news lead, learning how to handle quotes in a profile, learning how to frame the nut graf in a feature article, that kind of thing.  This kind of purposeful reading is extremely effective, and we need to learn how to incorporate it better into our regular English classes as well.

The Common Core’s emphasis on reading non-fiction should be in the “Writing” section, not the “Reading” section
What we don’t need is non-fiction just for the sake of non-fiction.  Unless kids are supposed to learn, immediately, about Mumbai slums or how to do amazing reportage, there’s no particular reason to have them read Behind the Beautiful Forevers instead of, say, Midnight’s Children.  I happen to prefer the Boo book to the Rushdie book, but a very gifted former student just wrote her college essay about how the Rushdie book changed her life.  That kid can write anything; all she has to do is spend a few hours immersing herself in some models (last year I suggested Pale Fire and a few weeks later she produced an astonishing Nabokovian story in the form of a scholar’s notes on a haiku).  For a particular writing assignment, she should read texts that can serve as models, but in general she should read whatever interests her.  There is good reason for her to read what interests her--or what interests her teacher--but there is no good reason for her to read what interests David Coleman, or an imaginary marketing executive.  The Common Core, like most curricula, is not founded on evidence or data.  In the matter of “Informational Text” as in many other matters, the Common Core is confusing the appearance of usefulness with the reality of usefulness, confusing what might be useful in the short term with what is necessary in the long term, and, above all, I suspect, showing a disdain for anything that is not ostentatiously practical. As so often, it seems to me that the people who see themselves as hard-minded realists are actually the ones who are deluded about reality, while those the realists deride as idealistic dreamers are actually a lot more realistic.

Over this, as I.B. Singer told my father-in-law about another tempest in a teapot, no children will die; but then, some children might suffer unnecessarily.  Why not just let kids read what they want to read?  The first thing is to provide time, space, and books, and the second thing is to create a culture in which books are discussed in a meaningful way.