US test scores are very good, when social class is taken into account (or, EPI continues to play the role of a graduate school of educational public health)

Richard Rothstein and Martin Carnoy of the Economic Policy Institute have a new report out that digs into the data from the 2009 PISA tests and says more authoritatively what already seemed clear to many of us from a cursory look at the data: in American schools that aren't plagued by poverty, students do very well on international tests.

Why this is not the standard line about education in the US
The truth is that it is inequality and poverty that drive educational issues, not the other way around.  You can't fix society by fixing education, and the only way to fix education is by fixing society.  Unfortunately, too many people have too much at stake in the way things are done now for anything to change.  In particular, no one wants to admit that we should directly tackle poverty, because that would probably entail, for the powerful, higher taxes and less power.  This may seem obvious, but when it plays out in real life the denial and dissonance it produces can be frustrating and confusing.

Forthright denial (a personal anecdote)
A couple of years ago, before I had thought much about this stuff, I went to a talk by Harvard's Tony Wagner at the MIT alumni club.  Wagner screened a film he had made about Finland; his argument seemed basically to be that American schools were terrible because we didn't focus on creativity and critical thinking, while Finnish schools led the world because they did focus on those things.  In the Q&A after the screening, I asked him about the role of poverty, and I specifically asked whether rich schools in the US scored as well as Finland.  To my very clear and specific question he answered a forthright and definite "No."

When I followed up by email, citing the same 2009 PISA scores that the new EPI report analyzes, Wagner backtracked, claiming that he had misunderstood my question--but then he asserted that tests didn't tell the whole story, and that Finnish schools, like certain progressive schools in the US (he loves the Parker school, for instance) are "worlds apart from even our best suburban high schools."  (Oh snap!  Take that, Leafstrewn!)  Still, I couldn't help but wonder why he had denied what was after all a pretty basic fact (in correspondence he said he was familiar with the data I'd cited).

Maybe frozen circumstances won't dance to any melody?
Wagner and I had an interesting correspondence for a while, and he was always polite and generous with his time, but I came to think that the way he answered, wrongly, my specific and clear question at the screening of his movie was a symptom of a deeper problem.  Basically, I found that while he was eventually able to acknowledge, in private correspondence, what was after all indisputable (poverty and social class are overwhelmingly important in academic performance), and while he was subtle enough to quote Karl Marx to support his rhetorical strategies, he simply could not have the kind of successful career he has enjoyed (appointment at Harvard, best-selling author, educational "leadership" guru, etc.) without downplaying the role of social factors like poverty and highlighting, instead, things like "creativity", "leadership," and "innovation."

Wagner himself more or less acknowledged this dynamic in the line he quoted from Marx. I had told him I liked a lot of what his book said about how to make schools better, but that insofar as the book imitated Thomas Friedman, I didn't like it (after a nod to Friedman's influence in the Acknowledgments page, Wagner started off the book, believe it or not, with an anecdote about a conversation with a CEO he happened to sit next to on a plane, and went on to couch its argument in Friedmanesque blither-blather about changing America's education system in order to compete in the 21st century global economy, etc.).  In response, Wagner wrote the following:

My favorite quote from Marx is: “To make the frozen circumstances dance, you have to sing to them in their own melody.”

... Unless and until we have an alliance of business and education leaders advocating for the kinds of profound changes that are necessary, I don’t think we have a prayer of a chance.

This is depressing, and not just because the Marx line is quoted out of context (the whole passage is about publicly shaming the powers that be, not letting them have even "a minute for self-deception and resignation," and making them "terrified"--all of which is very far from the aims of people like Tony Wagner and Thomas Friedman (1)). Wagner was basically saying that he can't acknowledge the truth about education, because that won't move the people who are really in power--"business and education leaders."  So in effect his position at Harvard, and his speaking at places like MIT, means that he cannot speak the truth--probably can't even remember the truth until some persistent naif like me sticks it indecorously in his face like a piece of soiled paper. The event I attended was held, appropriately enough, at the Broad Institute--funded by the same Eli Broad who has spent some of his billions on educational "leaders" (like Joel Klein) who are trying to institute a corporate, test-driven take-over that couldn't be further from the Parker school.

Where are our schools of educational public health?
All this is why we need graduate schools of educational public health--and why we probably won't get them.  But I'm happy to see the Economic Policy Institute, like many others (the BBA, David Berliner, Stephen Krashen, Diane Ravitch, and thousands of lesser known teachers and parents) stepping in to fill the void.
***********************************************************

Footnote:
 
(1) Here's the Marx passage that Wagner quoted: "Criticism dealing with this content is criticism in a hand-to-hand fight, and in such a fight the point is not whether the opponent is a noble, equal, interesting opponent, the point is to strike him. The point is not to let the Germans have a minute for self-deception and resignation. The actual pressure must be made more pressing by adding to it consciousness of pressure, the shame must be made more shameful by publicizing it. Every sphere of German society must be shown as the partie honteuse of German society: these petrified relations must be forced to dance by singing their own tune to them! The people must be taught to be terrified at itself ..."
(from "Towards a Critique of Hegel's Philosophy of Law")

Guys Read--but may not like the books they're assigned in high school!

A "Read In"
Today our school had a "Read In"--a whole day, in between midyear exams and the second semester, in which we shared books, read, and talked about reading. During one of my free blocks I was on a panel of four male teachers; we talked to an audience of about a hundred and fifty students about our own reading, we read to them from a book we liked, and we took their questions.

Even future teachers don't like assigned reading
In the course of the discussion, every single one of us let it slip that he had not liked much of the assigned reading in high school--that we liked reading, but not the books we were given in our English classes.  I remembered that the same thing happened on the panel I was on last year, and I asked a friend who was on another panel today, and he said he and the guys he was up on stage with had said the same thing.

Does high school discourage reading?
At a certain point in the discussion, when we were asked when we had really been turned on to reading, I turned the question back around to the audience and asked them to raise their hands if they read more now than they did in elementary school.  A few brave souls raised their hands.  Then I asked them to raise their hands if they read more in elementary school than they do now.  eighty percent of the audience raised a hand.

We need to change this
One day is a nice start, but we high school teachers really need to think about how we can change the way we encourage-- or discourage--student reading.  What we are doing now is not working very well.  I think the way to go is toward more independent reading and group discussion and analysis focused on shorter passages (and of course a lot of writing); in any case, what we are doing now, and what has been done at most high schools over the past fifty years or more, is not very inspiring.

One issue that might be studied by a professor of Educational Public Health

High school graduation rates have been going up steadily over the past fifteen years or so--why?

The obvious way to start thinking about that question would be to consider what's happened in schools.  That's one thing done by the Ed. School professor who's been studying the issue, Richard Murnane:

"The economist, a professor at Harvard’s education school, speculates that some high school students dropped out when high schools raised standards for graduation because they realized they wouldn’t get over the bar.

"The recent improvement, he speculates, may be the welcome byproduct of a upturn in math and reading skills, as measured by test scores, among minorities in the years before the students reach ninth grade."
 
Nevertheless, schools are probably the wrong place to look to explain changes in the graduation rate, since schools are not the main driver of their own success.  As Professor Murnane knows (he has himself done good work on the educational effects of changes in economic inequality), you'd be more likely to figure out why graduation rates are rising if you considered social factors, like a changing job market, changing family composition, or environmental clean-up or degradation.

As it happens, one out-of-school factor I saw mentioned was in a blog post that wondered whether the phase-out of leaded gasoline and the steep drop of atmospheric lead, which several scholars have credited with the steep drop in crime over the past twenty years, could also have contributed to the increase in graduation rates.

Why we need Graduate Schools of Educational Public Health

The leaded gasoline link is debatable and will no doubt be picked up by scholarly researchers who have more time and resources than bloggers do; perhaps Professor Murnane himself will look into it.  Unfortunately, Professor Murnane is an anomaly. Almost all of the research being done at the GSE seems to be focused on in-school factors. There are research projects on "leadership", on educational policy, on data, on "educational accountability", on child development, on teacher training, on teaching math, and so on.  Looking at the GSE's list of research projects is somewhat depressing, because none of the projects are focused on what would really make a difference in educational outcomes--that is, out of school factors like poverty, libraries, health, nutrition, and equality.

That's why I think we need a new graduate school focused on educational public health.  If only Bill Gates would do what the Rockefeller foundation did a hundred years ago, and put his money into something that would make a positive difference in educational outcomes, instead of funding largely irrelevant projects that will serve mainly to disempower teachers.

Educational Public Health and the "Broader, Bolder Approach"

A hundred years ago this year, in 1913, Harvard and MIT, in a joint venture, founded the first American school of public health.  That same year, the Rockefeller foundation sponsored a conference that led to the expansion of Harvard’s school of public heath and the founding of another school at Johns Hopkins.  Over the past century, the field of public health has made amazing progress—arguably, far more than medicine—in improving and expanding American lives.  Now, a hundred years later, it’s time for a similar effort in education.  Along with graduate schools of education, which focus mainly on in-school factors of our educational life, we also need graduate schools that would study the many significant out-of-school, society-wide factors in education. 

I wrote a post a few months ago outlining this idea of educational public health.  That original post is here.  I also got in touch with some of the very few Americans who work on these issues full-time.  Elaine Weiss, who runs an effort called the "Broader, Bolder Approach to Education," liked my post well enough to put an edited version of it up on her Huffington Post blog, under her and my byline.  You can find it here. 

The Broader, Bolder Approach, affiliated with the worthy Economic Policy Institute, is one attempt to do some of what I have suggested a graduate school of public education might do.  You can support them by visiting their website and reading about their work, and by joining luminaries like Linda Darling-Hammond and Diane Ravitch, and teachers like me, and signing on to the BBA's mission statement.

Today's YA books have easier vocabulary than those of the past

Books kids want to read
One of my daughter's friends gave her a Nancy Drew book for her eighth birthday, and I read it to her last month. My daughter enjoyed it and asked me to get her another one. So I stopped at a bookstore yesterday and bought my daughter The Secret of Red Gate Farm.  I got a book for my son at the same time: Jay Asher's popular YA novel, Thirteen Reasons Why.

After a few weeks of not much reading, both kids are racing through the new books. My daughter and I are on chapter seven of the Nancy Drew, and my son has already finished Thirteen Reasons Why. This reminds me, again, of how important it is for kids to get their hands, not just on books (both kids have dozens of unread books in their rooms), but on books they want to read (there's a reason the unread books are still unread). I've also, however, been interested by the contrast between the two books.

Hard Words; Easy Words
I read lots and lots of Hardy Boys and Nancy Drew books when I was a kid. Edward O. Stratemeyer knew what he was doing. I vividly remember the nonstop excitement, the melodramatic adverbs, the cliffhanger endings, replete with exclamation marks, at the end of every single chapter. What I didn't remember, until I started rereading the books with my own children, was how extraordinarily sophisticated so much of the vocabulary is. The books produced by the Stratemeyer syndicate (not only for the Hardy Boys and Nancy Drew series, but also Tom Swift, The Rover Boys, and many others), had simplistic plots, cardboard characters (every villain speaks in a "guttural" or a "harsh" voice, every villain gives himself away by "something shifty in his eyes", etc.)--but there is nothing simplistic about the level of diction in these books.

The ten pages my daughter and I read last night contained the following words: racketeers, chided, sleuth, torrent, blockade, treacherous, mishap, endeavor, traction, deluge, assuage, haggard, resume, adjoining, dismay, gingham, heave, defer, bustle, meringue, uppermost, dissuade, wince, incensed, radiant.

Twenty-five words that difficult in ten pages is quite a lot, and not just for an eight year old. My ninth grade students would have known few of those words. Curious, I went downstairs and picked up the Jay Asher novel that my twelve year old son is reading, a book I bought because it was a favorite of one of the seventeen year olds that my wife has been tutoring. I looked at the first ten pages of the book. Here are the only hard words I found: throbbing, sequence, serrated, meanders, encore. Five words, and significantly easier, on average, than the ones in the Nancy Drew.

Why do today's YA novels have such low-level diction?
I don't think Thirteen Reasons Why is unusual. Most of today's YA novels, though they are written in many other ways with a literary sophistication and subtlety that completely outclasses Nancy Drew, have very, very few words that would be unknown to even the most illiterate ninth grader. This is a pretty important problem for those of us who care about our students' learning words in the way that everyone agrees is the best: by reading.

Reading should be the best way to learn vocabulary, but it won't work if the books you're reading don't contain any words you don't already know.  It also won't work well if the rest of the book is too hard for you.  That's why more skillful readers have an advantage in learning vocabulary from reading: because more skillful readers can understand text in which more of the words are unknown to them, they are more likely to encounter more unknown words, and more likely to understand them when they encounter them.  As usual, the rich get richer.  This is why books like Nancy Drew are so wonderful--the books are really easy to understand, so weaker readers (or younger kids) can read them, but they have lots of sophisticated words in them, used in contexts that are often quite easy to figure out.

Besides John Green, what other YA authors use big words?
His sophisticated vocab is another reason to love John Green; the first ten pages of The Fault in Our Stars contains: abundant, redeeming, facet, preternaturally, recurrence, peril, disinterest, exotically, cannula, contraption, myriad, cankle, serenity, veritably, Episcopal, eking, meager, satellite, monotone, appendiceal, proffer, denounce, proverbial, oblivion, encompassingly. Twenty-five words, just as many as the Nancy Drew book, but used in a prose that's a million times more limber and elegant.  In fact, the romance in The Fault in Our Stars begins partly because the two protagonists impress each other with their fancy vocabulary.

So how do we get our students to read books that with more difficult words in them? How can we get more people to write like John Green?  And what other YA novels contain difficult words? (1)


*********************************************************************
 Footnote:
1. Lexile scores are helpful first takes, but aren't reliable.  Sarah Dessen's Someone Like You, for instance, has a lexile score that's higher than the early Nancy Drew books, but doesn't use nearly as many difficult words of the "good SAT word" variety: I count fewer than ten in the first ten pages.

Leafstrewn's technology deficit and the inadequacy of philanthropy

Leafstrewn is a great school in almost every way, and I wouldn't want to be teaching anywhere else.  Our technology deficit is real, however, and I've been wondering why.  Here's a quick post telling a story about one small piece of the issue.

Our technology is an embarrassment, but maybe that's okay? 
We don't have projectors in our classrooms.  There aren't many computers in the building for kids to use.  Our new platform for online assignments and grading was rolled out in September with pretty much no heads up to the faculty, and very patchy training.  For a district of our wealth and prominence, our technology systems are somewhat lame.  Maybe this is okay?  I can easily imagine that all the fancy technology in the world wouldn't improve reading scores.  (That's what John Hattie claims the data says.)

Our piecemeal attempts to reduce our technology deficit
Nevertheless, the way Leafstrewn manages things is pretty absurd.  Not long ago, I was asked by some colleagues to join them on a grant application to the Leafstrewn Education Foundation, one of a few private  organizations that support public education in Leafstrewn, to buy tablets and styluses on which we could grade electronically submitted essays.  Each tablet costs $85, and there are nine of us, so the grant is for $765.  I really like the people who were writing the grant, and I figured it would be interesting to try out a tablet, so I said I'd be happy to be on the grant.

For our application, we had to write up a proposal and then make a pitch before a six-person panel.  So I traveled to one of the elementary schools and waited around with five of the other people in our group for half an hour or so before we made our 15 minute pitch.

At the end of the pitch, one of the thoughtful and judicious members of the thoughtful and judicious panel said, "So, what if we couldn't give you the money to buy everyone a tablet.  Would it still be worth doing if you had to share them--if you only got five tablets, say?"  And of course our team said, "Oh, yes."

Even as we said this, we were thinking--or at least I was thinking: WHAT?!  We did all this for a shot at less than five hundred dollars?  I should have just fronted the money myself!

Maybe it was just the indignity of having to come cap in hand before a panel of (what I imagined were) one-percenters to beg them to buy me a little electronic toy that costs less than a hundred bucks, but I couldn't believe the inefficiency of the whole process.  So many people spending so much time for so little money--less than a hundred dollars per person-hour, just counting the hours put in by our team.  If you include the hours spent by the panel themselves (they seemed to take it pretty seriously), the whole exercise looks even more ridiculous.  The panel could have taken our grant applications, read them over, and decided, and nobody would have had to spend an hour of his Tuesday afternoon.

The inadequacy of philanthropy
I ended up wondering if this is what always happens when private money gets involved in public goods: inefficiency. If you want to help the schools, maybe you should just give the money to the central administration.  The Superintendent is hired to make the schools as good as they can be within a certain budget.  If you don't think the Superintendent is doing a good job, wouldn't it be better to try to lobby the Superintendent, rather than trying to do his job yourself in your spare time?  I can't help thinking the point is less to use the money to make the schools better (the funding from this organization and from the other philanthropic organization that funds special programs at Leafstrewn High adds up to about half a million dollars, which is a rounding error in the School Department's overall budget of about 90 million dollars) than something else, I'm not sure what.  To make rich people feel good about themselves?  To give people an excuse not to push for higher taxes?  I don't know, and I don't really get it.  But I do think that philanthropic giving in Leafstrewn, like philanthropic giving nationwide, is fairly inefficient, mostly self-serving and should probably not be tax deductible.
Post Script
In the end, we got the grant for the whole $765.  Maybe the tablets will be useful; I still suspect that private foundations for public schools are inefficient and undemocratic. Am I wrong?

US schools perform extremely well on international tests

US students do well; Massachusetts students far outperform Finns
This is worth noting, as a counter to the usual scare stories about the failure of American schools and teachers: US students did really, really well on the recent TIMSS (math) and PIRLS (reading) tests.  Many individual states did even better. In our state, Massachusetts, the average score on the 8th grade math test was far better than that of Finland, a country with a comparable population. In fact, despite a continuing gap between the scores of white students and black students, the average math score for African-American 8th graders in Massachusetts was better than that of the average for all 8th graders in Finland.  It's very likely that if reading scores were available for Massachusetts, our students would have done better overall than Finnish students (a state-level reading score was only available for Florida, and Floridians scored as well as the Finns).

Hooray!

Junkets to Leafstrewn?
Perhaps the most important thing about this news is the way it has been reported in the mainstream press (1).  As usual, it is very important to read press reports about education, and everything else, with a skeptical eye. Some papers, like the DEtroit Free Press that I linked to before, covered the story reasonably well. Others, like the Times and the Post, stuck largely to the "School Failure" storyline.  Here's the headline in the Washington Post, for instance: "U.S. students continue to trail Asian students in math, reading, science."  A better headline might be: "U.S. students perform extremely well on international tests".

You might think that the press might switch the narrative from "Our schools are failing, so we need reform" to "Look, reform is working"--but a meme as well-established as the "failure" of US schools has so much momentum that it is, like the Titanic, almost impossible to turn.  If enough icebergs like these results come in, maybe the old meme will finally sink, and education experts can take junkets to Leafstrewn instead of junkets to Finland.

Now if we can just eliminate poverty...
On the other hand, these scores also remind us that there are wide gaps between rich kids and poor kids, and the best thing we could do to improve our students' academic achievement is reduce inequality.  If the US had the poverty levels of Finland, just think how well we'd do then!

***************************************************************************
Footnotes:
1. Bob Somerby at the Daily Howler has done an excellent job of deconstructing the press coverage of these recent scores. 

An interesting question in regards to teaching

News flash: new research shows that if you run, you get better at running
To me it was news that this was ever in doubt, but a recent post on a New York Times blog (okay, not so recent, but a smart education professor just sent it to me) tells us there is a hot debate over whether running should be taught, and that scientists in England have done a study showing that runners develop more efficient form "just by running more."

Apparently the runners in the study increased their "running efficiency" (oxygen used at a given pace) by nearly ten percent over a ten week period in which they were given no instruction, they just ran.  The scientists say that the runners all learned to bend their knees and ankles more at certain appropriate points in their strides, and all learned to be less "wobbly in the rear foot," whatever that means. 

The post made me think, naturally, about reading; perhaps, like running, reading can be improved just by doing it? Hm...

An interesting question
The most eye-opening thing in the post was not the (somewhat predictable) result that regular running made people better at running, but a little paragraph near the end of the post in which the researcher who led the study said that the results ""raise an interesting question in regards to teaching people to run."

What was this interesting question?  The next sentence in the article began: "'If runners can self­-optimize,' as the women in this study seemed to do, then..."

Okay, how would you finish that sentence?  What interesting question does the study raise in regards to teaching people to run?  I myself thought the question was going to be: "Why bother?"

But no!  The interesting question about teaching running that this study raises for the researcher is not, "Why bother?" but instead: "maybe we should teach runners to learn to understand how the movement feels to them."

Ha!  This woman has done a ten week study showing that people can improve on their own, without instruction, and her conclusion is that maybe we should teach people to learn to understand how the movement feels to them?!

I mean, I am not against mindfulness.  Far from it.  I recently spent a weekend at a Buddhist monastery, and it was great!  Pay attention to the way things feel to you, by all means.  It will help you in many ways, and it might even help your running, and your reading.  But if non-teaching works well for some things, why not just leave it at that?

Interestingly, non-teaching doesn't work well for everything...
The runners in the experiment all learned to bend their joints more efficiently, and all learned to be less wobbly--so maybe those aspects of running aren't worth teaching. None of the runners in the study, however, learned to land on the middle or front of the foot, which some scientists (and, if I remember correctly, the running tribe in the book Born to Run) say makes the stride more efficient and the runner less injury-prone.  So maybe people do need to be taught that?

This result goes along with what I've been thinking about "Visible Learning" and "Direct Instruction" and so on.  There are very likely some things (math?) that require more explicit instruction, and some things (reading?) that require less explicit instruction, and even within a discipline, like reading or running, there may be some things that require more or less explicit instruction. 

One might think that this conclusion--use direct instruction when it's helpful and don't use it when it's not necessary--would be obvious to everybody, but it's not obvious.  We are, I hope, nearing the end of a counterrevolution against the progressive education of the seventies and eighties, and we have gone so far that people like the researcher in the running study cannot imagine anyone learning anything that is not taught--even when she has herself just completed a scientific study showing exactly the opposite.

Teaching or no teaching, you can't run better if you don't run at all
Again, no matter how we are teaching, we need to make sure kids are reading enough.  So I'll close with this old chestnut (table from Allington, numbers originally from here):

Reading Volume of Fifth Grade Students at Different Levels of Achievement

Achievement                         Minutes of                                       Words per

Percentile                            Reading per day                                     year

90th                                           40.4                                             2, 357, 000

50th                                           12.9                                                601, 000

10th                                             1.6                                                  51, 000

 

Rappelling and reading: When is "direct instruction" not appropriate?

Learning Targets: "Isn't this stupid?"
At my son's school, every Monday his ELA teacher hands out a syllabus with the daily learning targets for the week, and at the beginning of every class the teacher writes the day's Learning Target on the board.  This past week the Learning Targets were:

• I can write engaging introductions to draw my readers in.
• I can write strong topic sentences to guide my readers through my writing.
• I can reread my own writing to make choices for revision.
• I can eliminate unnecessary parts of my writing and add transitions to support my readers.
• I can add information and adjust my word choice to improve meaning and precision of language.

On the one hand, these seem like perfectly reasonable things for students to be working on. On the other hand, when my son just handed me the sheet, he said, "Isn't this stupid?"

True, my son is a twelve-year old to whom almost everything seems either totally stupid or totally amazing, and yes, he has imbibed some of my own skepticism about pretending that reading and writing can be broken down into discrete and teachable "skills"--but some of the most mature, sophisticated and enthusiastic of the high school juniors in my "Honors" American Literature class also seem to think "Learning Targets" are ridiculous. When I wrote a "Learning Target" on the board of my own classroom one day a couple of weeks ago, a few of the top students in the class, the kids who are taking three AP classes and are sons and daughters of fancy doctors or professors at Perfect University, said, "Are you kidding?"

That my son and my students are so critical and even dismissive of these Learning Targets is interesting, because Learning Targets are central to the way education is conceived of these days. A  lot of the current discourse about education says that teaching and learning  should always have specific goals ("Learning Targets"), that it should always be "visible" to teacher and student, and that student progress should be assessable along the way. This is often a good idea, but it has its limits. The reaction my son and my students had is largely related, I think, to the fact that it was in English class.  My guess is that if their AP Math teacher had put up learning targets (e.g. "I will be able to calculate the mean, the variance, and the standard deviation for a population or for a sample"), my students would have found it less risible.

"The heart of the model of successful teaching and learning"?
A prominent book on education research offers the following as an illustration of its central argument:

Consider the following: I am going to take you to the top of a three-storey building and teach you to rappel down the outside of this building.  Typically, I would demonstrate to you how to put on a safety harness, tie the rope in a bowline, and then show you how to lean backwards to commence the descent.  In line with the principles of of good teaching, I would then ask you, the student, to implement this learning.  Typically, such a learning situation lead to much care by the students, an enhanced level of interest in what peers are doing, and high levels of help-seeking behaviors to ensure the knowledge of rope-work is correct and harnesses are correctly positioned.  The goals are challenging, specific, and visible, and the learners are committed to them!  The learning is actively visible and there are high levels of feedback and monitoring.  The learner typically "seeks" the feedback.  When a novice first gets to the edge, there is a remarkably high level of peer teaching and learning: it is not natural to fall backwards when descending as it is more typical for the feet to precede the head.  When finally the student reaches the bottom there is a surge of excitement appreciating that the challenging goal has been reached (it is abundantly clear what the success criteria are!), the experience was exhilarating, and the learning absorbed in the experience itself.  Most want to repeat the experience and continue to enjoy meeting the challenging goals.  Moreover, all these acts and most of the "thinking" about the task are visible to the teacher and to the learner.  This is the heart of the model of successful teaching and learning advocated in this book.

The first time I read this passage, I laughed out loud.  The notion that rappelling down the face of a three story building is in any way comparable to what students do in school is just ridiculous. Nevertheless, I think looking at the passage a bit more closely may shed light on when this model of teaching and learning is appropriate and when it is not.

In the spirit of being visible and explicit, let me state the obvious:

(1) Rappelling can be taught in a day.  Even the basics of reading usually take a year or more.

(2) Rappelling is (as presented here) a new skill that the "novice" is trying for the first time.  Reading is for nearly all students not a brand new skill (see #1), and most English teachers, like me, are teaching kids who have been studying reading for more than five years.  I'd like to see the education researcher try to teach rappelling to people who have been doing it nearly every day for five years.  He might find that his model didn't seem quite as snazzy.

(3) Rappelling is, especially for the novice, hugely exciting.  Reading is pleasant, but not exciting.

(4) Rappelling is potentially extremely dangerous to your health.  Reading is fairly safe.

(5) If it is "abundantly clear what the success criteria are" in rappelling, the same is much less true in reading.  Even in an hour-long assessment, it is difficult to measure reading skill with much precision on an individual level, and in daily practice the success criteria are far from "abundantly clear."

(6) Rappelling has certain discrete and fairly straightforward skills and conventions (how to tie a bowline and put on a harness, etc.) that are necessary to master before you begin the difficult part.  In this, it is actually similar to reading.  Reading requires knowing some facts (the sounds the letters represent), knowing a few conventions (you read from left to right, etc.).  Mastering this knowledge is necessary but is, in some sense, the easy part.  But with both reading and rappelling, and perhaps with every important skill, the really important part is the practice--actually doing it.


Actually doing it
Although I have not rappeled, I have both learned and attempted to teach a very similar, and perhaps more difficult task: "dropping in" to a halfpipe on a skateboard.  Learning to drop in is quite similar to learning to rappel: there is the same frightening moment at the top of a vertical face when all of your instincts are telling you to lean back, not to lean forward, and telling you that if you are going to fall, you had better fall feet first.  As the passage says, "it is more typical for the feet to precede the head."  In some ways, dropping in may be even more scary, since there's no rope.  It's quite thrilling, even after you've done it thousands of times.  These photos can't convey the thrill, but the one on the right gives some idea of the scale that's often involved:

Now, what inevitably happens to beginners is what happened to me the first time I tried to drop in: you don't lean forward enough, and you slide down the ramp, kind of like this--except that this guy is leaning so far forward, he is clearly not a beginner:

When I dropped in for the first time, at a ramp in Canton, Massachusetts, in 1986 or so, I sprained my ankle quite badly. Would better instruction have made it easier for me?  I don't think so; my friend Jon had told me everything I needed to know, had modeled it for me, and had offered lots of encouragement. I have since tried instructing other people to drop in, and there's not much to say; the steps are quite simple:

(1) put your back foot, and all your weight, on the tail of the skateboard, which is perched on the coping of the ramp;

(2) put your front foot lightly on the front of the skateboard;

(3) simultaneously push down hard with your front foot and throw all your weight forward and down.

That's it.  There is no knot-tying, no complicated safety harness (everyone knows how to put on a helmet and knee-pads).  It's all in the doing. That last step, pushing down hard with your front foot and throwing all of your weight forward and down into the ramp, is both the most important and the most terrifying.  And it famously cannot be taught. An online how-to has this to say about dropping in:

It can be scary to stomp down and lean into open air. There is no turning back once you've started the stomp, and I would say at least 80% of the problems people have when dropping in is not being committed enough to this part. You have to trust that you and your skateboard will make this work. You have to invest in dropping in 100%. It's all or nothing. Be committed to the drop in. Once you do it, it will get easier and easier every time.

Here's a secret about skateboarding - skill is very important, but even more important than skill is self confidence. It's all in your head. This is what separates something like skateboarding from other "sports". Your strongest opponent is yourself. So when you face something like dropping in, and you do it, you are taking a huge step toward self control.

That was a little deep, but it's true. The point is, if you are going to try and learn to drop in, then just do it. It's like Yoda says, "Do or do not, there is no try."

Just do it.  When my brother and I had a half-pipe in our backyard, and when I was the advisor of the skateboarding club at Leafstrewn, I noticed this about skateboarding: there was a strong culture, not only of "just doing it", but of encouragement, support and collaborative learning.  The important things were not the direct instruction or the "clear learning goal," but the culture, the engagement, the modeling, feedback and encouragement from peers, and the endless practice.

 Direct Instruction? Yes, in small doses.
"Direct instruction" is a pretty good way to teach people a new skill.  You make it clear what you're about to teach and why, you teach it and model it, you have students practice it under your supervision, you review, and then students practice it on their own.  These steps seem like common sense.  If I were going to teach people how to, say, fix a flat tire on a bicycle, this is pretty much how I would do it.  Nevertheless, this is not to say that all of school should consist of "guided instruction."

In learning rappelling, or dropping in, or reading, direct instruction should play a role--but only a small role.  There may be subjects, like math or science, in which there should be a new learning target every day, and there should be new instruction every day, but English is mostly not like that.  The most striking thing about the learning targets my son's 7th grade teacher put on the board this past week is that they might serve just as well for my 11th grade students.  In reading, the learning goals are even less "challenging, specific, and visible" than they are in writing, and the success criteria are even less "abundantly clear."

Therefore, even though I am trying to be clearer about what the learning goals are in my classes ("I will become more comfortable reading colonial-era literature"; etc.), and even though I am trying to be clearer and more deliberate in my instruction, I still think the most important thing is to foster a culture of engaged reading, humane discussion and intellectual inquiry.  There is a mania in education today for direct instruction, visible learning, abundantly clear learning targets, accountability, testing, and so on.  These things are important, but there is a great danger that we will go too far and think that every minute of every schoolday must be specifically targeted, and that the only worthwhile learning targets are those with abundantly clear success criteria.

As always, we need to think about the long term.  After trying it over and over, I did finally learn to drop in, and it was fun--as learning should be.

Can we trust educational research? ("Visible Learning": Problems with the evidence)

I've been reading several books about education, trying to figure out what education research can tell me about how to teach high school English. I was initially impressed by the thoroughness and thoughtfulness of John Hattie's book, Visible Learning, and I can understand why the view of Hattie and others has been so influential in recent years.  That said, I'm not ready to say, as Hattie does, that we must make all learning visible, and in particular that "practice at reading" is "minimally" associated with reading gains.  I discussed a couple of conceptual issues I have with Hattie's take in an earlier post--I worry that Visible Learning might be too short-term, too simplistic, and less well-suited to English than to other disciplines.  Those arguments, however, are not aimed at Hattie's apparent strength, which is the sweep and heft of his empirical data.  Today, then, I want to address a couple of the statistical weaknesses in Hattie's work.  These weaknesses, and the fact that they seem to have been largely unnoticed by the many educational researchers around the world who have read Hattie's book, only strengthen my doubts about the trustworthiness of educational research.  I agree with Hattie that education is an unscientific field, perhaps analogous to what medicine was like a hundred and fifty years ago, but while Hattie blames this on teachers, whom he characterizes as "the devil in this story" because we ignore the great scientific work of people like him, I would ask him to look in the mirror first.  Visible Learning is just not good science.

Hattie's data
Visible Learning attempts to be both encyclopedia and synthesis. The book categorizes and describes over 800 meta-analyses of educational research (altogether, those 800 meta-analyses included over 50,000 separate studies), and it puts the results of those meta-analyses onto a single scale, so that we can compare the effectiveness of the very different approaches.  After categorizing the meta-analyses, into, for instance, "Vocabulary Programs", "Exposure to Reading", "Outdoor Programs", or "Use of Calculator", Hattie then determines the average effect that the constituent meta-analyses show for that educational approach.  By these measures, exposure to reading seems to make more of a difference than the use of calculators, but less of a difference than outdoor programs, and much less of a difference than vocabulary programs. (There are some odd results: "Direct Instruction," according to Hattie's rank-ordering, makes more of a difference than "Socioeconomic Status.")

Like other teaching gurus and meta-meta-analyzers (for instance, Robert Marzano, whose 2000 monograph, A New Era of School Reform, makes the case very explicitly), Hattie believes that good teaching can be codified and taught (that sounds partly true to me), that good teaching involves having very clear and specific learning objectives (I'm somewhat doubtful about that), and that good teaching can overcome, at the school level, the effects of poverty and inequality (I don't believe that).  Hattie uses a fair amount of data to back up his argument, but the data and his use of it are somewhat problematic.

First, questions about the statistical competence of Hattie in particular
I am not sure whether we can trust education research, and I am not alone.  John Hattie seems to be a leading figure in the field, and while he seems to be a decent fellow, and while most of his recommendations seem somewhat reasonable, his magnum opus, Visible Learning, has such significant issues that my one friend who's a professional statistician believes, after reading my copy of the book, that Hattie is incompetent.

The most blatant errors in Hattie's book have to do with something called "CLE" (Common Language Effect size), which is the probability that a random kid in a "treatment group" will outperform a random kid in a control group.  The CLEs in Hattie's book are wrong pretty much throughout.  He seems to have written a computer program to calculate them, and the computer program was poorly written.  This might be understandable (all programming has bugs), and it might not have meant that Hattie was statistically incompetent, except that the CLEs Hattie cites are dramatically wrong.  For instance, the CLE for homework, which Hattie uses prominently (page 9) as an example to explain what CLE means,  is given as .21.  This would imply that it was much more likely that a student who did not have homework would do well than a student who did have homework.  This is ridiculous, and Hattie should have noticed it.  But even more egregious is when Hattie proposes CLEs that are less than 0.  Hattie has defined the CLE as a probability.  A probability cannot be less than 0.  There cannot be a less than zero chance of something happening (except perhaps in the language of hyperbolic seventh graders.)

As my statistician friend wrote me in an email, "People who think probabilities can be negative shouldn't write books about statistics."

Second, doubts about the trustworthiness of educational researchers in general
My statistician friend is not the first to have noticed the probabilities of less than zero. A year and a half ago a Norwegian researcher wrote an article called "Can We Trust The Use of Statistics in Educational Research" in which he raised questions about Hattie's statistical competence, and in follow-up correspondence with Hattie the Norwegian was not reassured.  (Hattie seems, understandably, not to want  to admit that his errors were anything more than minor technical details.  In a exchange of comments on an earlier post on this blog, as well, Hattie seems to ignore the CLE/negative probability problem.)

For me, the really interesting thing about Hattie's exchange with the Norwegians was that he seemed genuinely surprised, two years after his book had come out, by the fact that his calculations of CLE were wrong.  In his correspondence with the Norwegians, Hattie wrote, "Thanks for Arne Kåre Topphol for noting this error and it will be corrected in any update of Visible Learning."  This seems to imply that Hattie hadn't realized that there was any error in his calculations of CLE until it was pointed out by the Norwegians--which means, if I'm right, that no one in the world of education research noticed the CLE errors in between 2009 and 2011.

If it is true that the most prominent book on education to use statistical analysis (when I google "book meta-analysis education", Hattie's book is the first three results) was in print for two years, and not a single education researcher looked at it closely enough and had enough basic statistical sense to notice that a prominent example on page 9 of the book didn't make sense, or that the book was apparently proposing negative probabilities, then education research is in a sorry state.  Hattie suggests that the "devil" in education is the "average" teacher, who has "no idea of the damage he or she is doing," and Hattie approvingly quotes someone who calls teaching "an immature profession, one that lacks a solid scientific base and has less respect for evidence than for opinions and ideology" (258).  He essentially blames teachers for the fact that teaching is not more evidence-based, implying that if we hidebound practitioners would only do what the data-gurus like him suggest, then schools could educate all students to a very high standard.  There is no doubt that there is room for improvement in the practice of many teachers, as there is in the practice of just about everyone, but it is pretty galling to get preachy advice about science from a guy and a field who can't get their own house in order.

Another potential problem with Hattie's data
Aside from the CLE issue, I am troubled by the way Hattie presents his data.  He uses a "barometer" that is supposed to show how effective is the curricular program or pedagogical practice he is considering.  This is the central graphic tool in Hattie's book, the gauge by which he measures every curricular program, pedagogical practice and administrative shift:

Note that developmental and teacher effects are both above zero. What this implies is that the effect size represented by the arrow is not the effect as compared to a control group of students that got traditional schooling, nor even the effect size as compared to students who got no schooling but simply grew their brains over the course of the study, but the effect size as compared to the same students before the study began.

This would imply that offering homework, with a reported effect size of .29, is actually worse than having students just do normal school, or that multi-grade classes, with an effect size of .04, make kids learn nothing.

Now, that is obviously not what Hattie means.  The truth is that Hattie sometimes uses "effect size" to mean "as compared to a control group" and other times uses it to mean "as compared to the same students before the study started." He seems comfortable with this ambiguity, but I am not.  Not only is the "barometer" very confusing in cases like homework and multi-grade classrooms, where the graphic seems clearly to imply that those practices are less effective than just doing the regular thing (especially confusing in the case of homework, which is the regular thing), this confusion makes me very, very skeptical of the way Hattie compares these different effect sizes.  The comparison of these "effect sizes" is absolutely central to the book.  Comparing effect sizes (and he rank orders them in an appendix) is just not acceptable if the effects are being measured against dramatically different comparison groups.

Hattie, in a comment on an earlier post in which I expressed annoyance at this confusion, suggested that we should think of effect sizes as "yardsticks"--but in the same comment he says that effect size is the effect as compared to two different things.  In his words: "An effect size of 0 means that the experimental group didn't learn more than the control group and that neither group learned anything."  Now, I am an English teacher, so I know that words can mean different things in different contexts.  But that is exactly what a yardstick is not supposed to do!

Of course, it is possible that many of Hattie's conclusions are correct.  Some of them (like the idea that if you explicitly teach something and have kids practice it under your close observation, then they will get better at it more quickly than if you just ask them to try it out for themselves) are pretty obvious.  But it is very hard to have much confidence in the book as a whole as a "solid scientific base" when it contains so much slipperiness, confusion and error.

Beyond these broad issues with Hattie's work, I also have some deep qualms about the way he handles reading in particular.  Maybe one day I'll address those in another post.

David Coleman's beautiful fantasy: that students will read 7000 pages a year!

In a Boston Globe column cheering on the Common Core standards and their infamous call for 70% of student reading in high school to be non-fiction, David Coleman is quoted as saying that "non-fiction will mostly be read in science, history, and social studies classes, where students ought to be using primary sources and learning to decipher scientific research."

This is pretty interesting. I'd be happy if students read as much fiction as they are supposed to do now but read twice as much nonfiction, but that is very obviously not going to happen. The problem with reading a lot is that it takes time, and if student reading is really split 70-30, then Leafstrewn students who are now assigned 8 novels a year (2000+ pages) would then have to read the equivalent of close to 7000 pages--which would probably be great, but would not leave a lot of time to work on all the other stuff that's mentioned in the standards--let alone leave time for students to do a lot of independent reading--another thing Coleman has on occasion called for. 

Another possibility is that the CC people envision much more independent reading, and much less whole-class reading than is currently assigned, perhaps because they realize that a lot of students don't do the reading.  That's a vision I could respect, but I don't think David Coleman and his collaborators have thought it through that deeply. Instead, my sense is that, notwithstanding the Oxford degree the Globe columnist admiringly cites, Coleman has little idea what it's like in an actual school and little sense of how the document he was paid to prepare might actually be put into effect.

Student stories about reading

I had my 9^th graders write a little essay about reading.  I gave them a Malcolm X passage about how reading was liberating, and asked if reading was liberating or confining for them now, how it had changed, and what they thought it would be like in the future.  I encouraged them to include specific details, but I tried not to tell them what to write.  

One common theme was hating being forced to read:

“Up till I was in about third grade my parents forced me to read books like the ABC Mystery books.  I hated it because I didn’t pick them out of interest, I read them because I was forced to.  I also hated reading because it was hard for me growing up.  I can remember my second grade teacher Ms. D--- helping me read my first chapter books.  But as I got older I started to pick my own books and started reading and liked some books.”

“Unlike Malcolm X I don’t feel free when I read.  Every time I have to read I am being forced either by a teacher like Mr. C--- or by my parents.  I can’t recall one time where I have ever wanted to read voluntarily.  One of the reasons I hate reading is because I don’t picture the scenes in the book all I see are a bunch of words.  Reading maked me feel very trapped, almost as if being thrown in the Concord jail is a better place for me to be.  Reading also gives me a headache.  All of the makes me feel like reading is very confining.”

On the flip side, another common theme was the way other people—especially teachers—can influence kids’ reading in a very positive way:

“I think that overall I am a pretty strong reader and even though it can be boring at times I still think it’s fun.  Earlier this year in Ms. A---’s Honors class I read “The Odyssey, by Homer.  The book was long, confusing, and even boring for parts of it, but I still kind of enjoyed it.  I thought it was pretty interesting when Odysseus and his men got trapped in the Cyclops’ cave, and they had to trick him into getting drunk so they could stab him in his one eye to blind him, so that they could escape more easily."

"Reading is very relaxing because it allows me to step outside of everything I was doing before I started reading. […] I remember a couple of weeks ago when Ms. K--- showed me where And Then There Were None by Agatha Christie was in the mystery section of the library.”

“I remember when I never used to like reading.  Every time someone would ask me to pick up a book, I refused, or grabbed it and stared at it.  Just the sight of a book would make me sick, but not anymore.  It happened one day when I came back from school, I saw my older brother in the dining room reading a book.  He seemed genuinely interested in it.  The book happened to be Twilight, that was the book everybody was talking about then.  Before he even got to finish the page he was on I stole it from him and read chapter after chapter each passing day.  My fifth grade teachers had to beg me to put it down.  When I went home that night my oldest brother had brought home Twilight on Red Box and he said he would only keep it for one day.  So as the hours passed and the day turned to night, I was still struggling to finish the book so I could watch the movie.  Finally I just placed my bookmark in and set the book aside.  I popped the movie in the DVD player and settled back into the couch as the previews flashed by.”

“Right now I am reading Moneyball, by Michael Lewis, and it is one of my favorite books so far.  I saw the movie and I was recommended by my summer baseball coach, Mr. N---, to read it.  He told me that it was really good and he thought I would enjoy it because we both really liked baseball.  I remember that he recommended it to me at Eagle’s Deli after one of our summer baseball games at Warren Field.”

“Even though right now I say that I love reading, I used to hate reading.  It all changed when I met this person.  Until 5^th grade before I met this person, I didn’t really have any interest in reading.  I though it was very confining.  Once, when I was in the second grade, my mom almost spanked me because I didn’t read.  (Unlike the United States, many parents spank their children in Korea.)  It all changed when I met Ms. C----, my 5^th grade teacher.  When I first walked into her class, all the students stared at me because I was a new student.  I had difficulties with English.  She always helped me to improve my English.  I still remember the first time she handed me this book.  It was one day in winter, and we were released early from school because it was snowing so much.  I was waiting for my guardian to pick me up.  When I was doing my homework, she came up to me and smiled at me.  I didn’t know what she meant by smiling.  She said, “I have a present for you, A----. It is my Christmas present for you.”  I was very confused.  Then, she handed me a book.  It was a small book with a black cover, and the title of the boook was The Giver.  Looking at the old man in the front cover, I thought it would be boring.  When I got back home, I started reading the book.  That book with the old man on the cover changed everything about reading.  It made me feel liberated, unlike other hard books.  I felt like I was in the book with the characters.  I could even see their faces in my head.  Thanks to Mrs. C--- and a snow day, she opened a new world for me.”

What the heck does John Hattie mean by "effect size"?

OK, I am kind of annoyed.  I've been going through John Hattie's impressive book, Visible Learning, trying to make sense of it, and I'm confused on a basic point. Here I am reading this long book that is all about comparing effect sizes, and I realize that I don't know exactly what Hattie means by "effect size." If a program has an "effect size" of 0, does that mean that the the kids in the program didn't learn anything, or does it mean that they didn't learn anything more than the kids in the control group?  At different times in the book, it seems to mean different things. It is hugely annoying that Hattie doesn't make this clearer.  He claims to have spent 15 years writing the book, and he can't even be clear about the most basic possible element of his work?  Grrr.

Of course, part of what makes me so annoyed is that I am aware that the problem is probably partly mine as well as Hattie's.  If I were smarter, maybe I'd be able to figure it out.  But I can't help suspecting that maybe "effect size" means different things for different studies, and that this may have some effect on Hattie's conclusions.

Update (12/16): OK, it's not my problem, it's Hattie's.  More later, but this is just ridiculous.  The state of educational research is just pathetic.  This guy is some kind of big shot, and his book is supposedly a masterwork fifteen years in the making, and he's making various basic errors.  I didn't catch them at first because I'm an English teacher, not a statistician, but it didn't take me THAT long, either.  How is it that people continue to take him seriously?  I'll explain in a bit more detail in another post when I have a free half hour or so.

"Visible Learning," School Reform, and In-school Reading

(I wrote this really quickly; I apologize for typos.  I wanted to get it written while it was still fresh in my mind, but I have too much schoolwork to take a long time and write it well.)

In-school reading makes parents uncomfortable
At my son's middle school, and indeed throughout the district in the city my family lives in (slightly less leafstrewn than Leafstrewn, but graced with the Perfect University and the Institute of Machine Perfection), English teachers are having their students do a lot of in-class reading.  On Parents' Night, I told my son's teacher I was thrilled about this.  Last week, at a pot luck dinner for my daughter's class, a mom who also has a seventh grader told me she had wondered about what I said.  "You said, 'I think in-class reading is a great idea,'" she said. "I couldn't tell if you were serious,  or being sarcastic."

It turns out that she, and many other parents, see reading in class as a strange idea.  That same week, on a parent email forum, a couple of parents raised the issue: why, they said, does school have them doing silent reading, an activity that they could easily do at home?

I wrote a response in which I made the obvious arguments: that reading is (as even such a champion of whole-class direct instruction as Doug Lemov argues) always a productive use of class time; that it is, in fact (as Lemov says), the (very high) baseline against which any other use of class time should be measured; that many kids don't read at home and indeed hardly read at all; that in-class reading allows the teacher to do more one-on-one instruction, which is the best kind; and so on.

Of the parents who responded, most backed me up.  Some parents, however, were still not convinced. Most of these parents were willing to admit that reading in class might be good for other people's children--the implication I think was that good parents would have kids who read--but not for their own children, because their own children didn't need it.  At the pot luck dinner I talked to a couple of parents whose kids read a lot at home; they were confused as to why their kids would spend time in school reading when they could be doing something else.

What the Research Says
I spent several hours over the weekend immersed in John Hattie's magisterial 2009 overview, Visible Learning.  I've also spent some time looking back at Marzano's interesting 2000 monograph, A New Era of School Reform, which is, while less impressively encyclopedic, very much along the line that Hattie takes. Hattie and Marzano stress very clear and well-defined "learning objectives," very clear teacher control over where the lesson is going, and lots of "feedback" going both ways between teacher and student.  Most education reformers over the past twenty years or so have justified their efforts by pointing to research like that which Hattie and Marzano synthesize. For a different perspective, I also looked back at Stephen Krashen's summary of studies of in-class reading, The Power of Reading.  I've learned a lot, but I come away with more questions than clear answers.

Hattie and Marzano are very clear on what does work.  Hattie calls it "visible learning"--he says that it's very important that both teachers and students be able to see what is working and what is not working.  That requires very clear learning objectives, something Marzano stresses that seems to be taken to heart by all of today's reformers.  It also requires a lot of feedback, from teachers to students, but also, and according to Hattie even more importantly, from students to teachers. Teachers need to know what their students are "getting" and what they are not getting.   Teachers should be "activators" and not "facilitators."  Hattie argues strongly and explicitly against the classic "constructivist" conception of teaching, in which the teacher facilitates "authentic" learning that is open-ended, "student-centered," and aimed at "discovery"--that is, learning whose end points or goals neither the teacher nor the student know beforehand.  According to Hattie, this kind of "hands-off" teaching and "intrinsically motivated" learning is "almost directly opposite" to the kind of learning that the experimental data says is most successful.

Hattie and Marzano back up their arguments with vast amounts of experimental data.  Hattie's book is particularly sweeping: his book is a meta-meta-analysis--that is, a meta-analysis of over 800 meta-analyses, each of which included several individual studies.  Hattie's book synthesizes research on over a hundred million students, and with this vast body of evidence, a logical organization, a thoughtful, placid style but a forceful argument, it is certainly the most impressive analysis of educational data I've ever seen.  That said, I came away not fully convinced.

Two Ways to Critique Hattie: theoretically and empirically
There are two kinds of potential problems with the "recipe" that Hattie (along with Marzano and all the other proponents of "clear learning objectives") is selling.

One kind of problem is theoretical: it seems possible that this kind of learning is potentially dry, uninspiring, and limited--and perhaps especially boring to students who get the idea quickly;  also, it's not clear to me that, in the case of reading, the "learning goals" can be, to use Hattie's words, "challenging, specific and visible" (25).

A second class of problems, that I'll deal with in a post later this week, have to do with the empirical evidence Hattie is using: (a) the evidence for this recipe's efficacy seems to be quite thin in the case of reading; (b) the evidence for this recipe's efficacy seems to be mostly short-term, not long-term; (c) Hattie's evidence is mostly from controlled experiments, not from natural experiments; (d) Hattie and especially Marzano argue loudly that their evidence shows that poverty can be overcome by good schools and good teaching, but as far as I know this has never actually been done by an actual school in the real world--or at least not in the United States--despite the fact that thousands of schools across the country are trying to put this kind of "visible learning" into practice.

A theoretical critique: Is "Visible Learning" possible in English class--and if so, is it doomed to be dry and boring?
Today I wrote the following on the board in my "Honors" American Lit. class: "Learning Targets:  (1) I will be able to write a good question about a sophisticated text and find a specific passage through which I can explore the question; (2) I will be able to discuss coherently the relationship between Hawthorne and Transcendentalism."

The cluster of A students who sit near the front of the room noticed what I had written and hooted derisively.  One of them asked, "Is that a joke?"  Another said, "No teacher in my entire school career has ever written learning targets on the board."  Another one said it reminded him of a description of a typical day at a Fall River charter school that was posted a few days ago on Edushyster, an anti-corporate Ed reform blog.  This led to a spirited discussion for a minute or so, before I shut it down so that we could get to our learning targets.

One student said that she liked the idea of learning targets, even in English class, but the most common knee-jerk response from the kids was that putting up a "learning target" every day in English class was laughable and ridiculous. Again, we didn't spend much time on this, but I believe their reaction was based on two ideas that I've written about before: (1) that, on the micro level, the skills we work on in English class are very complex and interdependent, so that isolating one of them each day is either absurd (today we're learning about commas in Hemingway) or obvious (the one I put up), while on the macro level the skills are  (2) daily learning goals promote short-term thinking, while English-class skills are gained over the very long term--learned through repeated practice over years.

What are the learning goals in the case of upper-grade reading?
It is worth asking, then, what people who support "visible learning" would say should be our learning goals.  The obvious place to look for these learning goals is in the Common Core.

The Common Core has two reading strands for grades 6-12, one covering "Literature" and one covering "Informational Text."  I looked at the Literature strand.  The Common Core specifies 9 standards for literature in the ninth and tenth grade.  Three of them are below:

CCSS.ELA-Literacy.RL.9-10.2 Determine a theme or central idea of a text and analyze in detail its development over the course of the text, including how it emerges and is shaped and refined by specific details; provide an objective summary of the text.

CCSS.ELA-Literacy.RL.9-10.4 Determine the meaning of words and phrases as they are used in the text, including figurative and connotative meanings; analyze the cumulative impact of specific word choices on meaning and tone (e.g., how the language evokes a sense of time and place; how it sets a formal or informal tone).

These standards seem fairly unhelpful. They are essentially the same as standards from earlier grades (1).  That would be fine in itself; as I've written, English class is a matter of practicing the same skills over and over.  The problem is that these standards don't tell us anything we don't already know.  We know that every English student in the world will be required to "determine the meaning of words and phrases as they are used in the text." This is quite a low-level goal. 

In the end, I don't think putting a "learning target" up on the board is a particularly bad idea.  I don't think it's necessary, but in the hands of a decent teacher it won't necessarily hurt.  The danger with the practice is that it may lead to reductive, simplistic, boring teaching, in which kids are taught a lesson completely focusing on the simple structure of Hemingway's sentences.  On the other hand, the danger with the opposite approach, the let's-just-read--and-talk-about-it approach, is that it may lead to a class that is way too loose.  Either way is dangerous, but it seems unlikely that telling kids specifically what they are going to learn in every lesson is going to lead to more learning overall.  Learning in English class is, as the Common Core standards admit both implicitly and (sometimes) explicitly, cumulative and long-term.  To pretend that students are going to be noticeably better at "determining the meaning of words and phrases as they are used in the text" after a day-long lesson, or even after a month-long unit, is silly.  Determining the meaning of words and phrases is a life-long task--and it is one at which the learning curve becomes flatter and flatter.  I am probably better in some ways at this now than I was when I was twenty-five; but I'm not sure anyone would notice the improvement.  Does this mean that reading a new author (as when I discovered Edward St. Aubyn last year) is useless?  No, of course not.  But it would be silly to set targets for that learning ("I will be able to describe Patrick Melrose's family dysfunction...").

That may mean that more advanced students will be the more dismissive of "visible learning"; perhaps, but I think it's not necessarily great for any students, and I'll talk more about that in a post on the empirical critique of Hattie's work later this week.

********************************************************************************
Footnote:
(1)
The fourth grade standards include the following:


CCSS.ELA-Literacy.RL.4.2 Determine a theme of a story, drama, or poem from details in the text; summarize the text.

CCSS.ELA-Literacy.RL.4.4 Determine the meaning of words and phrases as they are used in a text, including those that allude to significant characters found in mythology (e.g., Herculean).

Do students need to "build reading perseverance"?

I was at a Metco conference all day today, and one of the workshops I attended was on the Common Core.  It was interesting in lots of ways that I'll have to mull over for a while, but one minor thing I was struck by was the way the presenter talked about independent reading.  The presenter, a high-school principal, told the story of a teacher in her school who twice a week puts high-interest YA books on every desk, and for the first 30 minutes of the period has the students just read.  When one of the students was asked why he thought they were spending 30 minutes twice a week in reading, the kid said, clearly repeating a phrase the teacher had instilled in him, "Well, we're going to have to take the SATs, so we have to build up our perseverance."

Later in the workshop, the presenter put up a slide of "tips" for ways ELA teachers can prepare their students for the Common Core and the tests that will stem from it.  One of her slides was titled "FOCUS ON READING", and it had two "tips". One was about reading in the content areas and the other was this:

"Students need to build reading perseverance"

I wasn't sure what I thought about this.  On the one hand, I was happy to see teachers encouraged to have their students do in-class reading of pleasurable texts--particularly in connection with the Common Core, which does not, as far as I can see, pay much explicit attention to reading volume or making sure that students are actually reading.  On the other hand, I was like: What?!  Perseverance?!  That is so depressing!  We "persevere" in things that are difficult, painful, discouraging.  Reading a good book is not supposed to be difficult, painful and discouraging; it's supposed to be entertaining, engrossing, FUN!

Maybe I'm just lucky, but my experience is that for well over 90% of my students (whether in the Metco program or not), they don't have much trouble sitting and reading silently.  This year, I had one girl who complained the most about how much she disliked reading; over the past couple of weeks I have had to tell her more than once, when we were having a discussion or doing a writing activity, to stop reading and put her book away.  Reading is fun--that's what we need to be helping our students to discover.

More on other stuff later; it was an interesting day.

Teachers, Instruction, Relationships

(A quick post--I'm thinking through these issues--the brain surgery analogy makes sense to me, though it may come out of left field for some other people... Also, I'm not sure why the font switches size or style--I wrote it all out at the same time, I think!)

Teachers are not brain surgeons--we are much more important...
Teachers are not brain surgeons.  I sometimes wish we were.  Surgeons get paid a lot, and surgeons get to feel like they're DOING something, get to feel like there is a clear problem and they are going to fix it.  Plus, their job is a lot easier than the jobs of regular doctors or of teachers, because they don't have to deal with difficult people.  The people they deal with are easy to deal with, because they are asleep.

While we teachers may wish we had anaesthetists to put our students to sleep before we worked on them, and while we may wish that what made the difference was the actions we ourselves did directly, rather than those we helped other people do, our lives are not so simple.

A lot of teacher discourse sounds like we are like surgeons, focusing on what the teacher does to, not with, the students, just as much of the discourse about reading education avoids the term "reading" and relies heavily on terms like "instruction",  "skills", "strategies", and so on. These terms--like the term "teaching" itself--make it sound like we are either conveying information in a gradgrindian way (filling little vessels with facts!) or else instructing children in a step-by-step process, as if we are ambulatory instruction manuals.  

But English teachers are not instruction manuals.  What's ironic is that just as teachers in other disciplines are getting away from "instruction" by, for instance, flipping the classroom, so that the teacher works instead as a coach, we English teachers, who arguably have less to "teach" than teachers in "the content areas", are constantly being told about "instruction."

Why would we want to be instruction manuals?  Instruction manuals don't actually work very well, because most people don't like to use them. Most instruction manuals, like the one in the John Ashbery poem, lead not to understanding, but to our dreaming of Guadalajara, or of some other place we have never been.  

Teachers are neither surgeons, operating directly on the brains of etherized students, nor ambulatory instruction manuals, mechanically dispensing information.  We are people dealing with other people, and the important things we do are to create conditions in which kids will want to read and write, and to help kids pay attention to their own reading and writing.  This may involve instruction, but often it does not.

Relationship before task
I once peeked at the "Course Expectations" a wise colleague of mine was handing out on the first day of the year.  My colleague told his students that when people were doing difficult tasks, studies had shown that they could do them better and longer if there were only another person in the room.  His goal that year, he had written to his students, was to be, for them, that "other person in the room."

I thought about my colleague's words when I was talking with my wife the other day.  My wife has been doing a fair amount of SAT tutoring recently, and she tells me that parents often talk as if what she will be doing with their children is sharing with them the "strategies" and "tricks" that she learned in her time as a high-achieving student at ivy league colleges and her stints as a college instructor at elite universities.  The truth is, she said, that when she and her students work on the reading sections of the SAT, what she mainly does is sit with the students while the students figure it out for themselves.  What I really do, she said, is to help them to see their own thought processes.  In order to help them do so, she needs to build a strong relationship with the student--so she always spends some time chatting with her students about what's important to them, what's going on in their lives.

My department chair is right to be fond of the dictum, "Relationship before task."  The task is the responsibility of the students; the relationship is our responsibility. 

Building Relationships (more wise colleagues!)

In today's department meeting, we were treated to a presentation by a couple of our colleagues who have been systematically investigating the ways in which they build relationships with their students.  Relationships with students, they said, are built through the many small moments, the micro-interactions, that we have each day with individual students.  They offered a set of helpful guidelines for handling these little moments, many of which may present themselves to us as difficult or annoying.  Many of these guidelines were familiar from parenting books I've found helpful in dealing with my own kids at home (acknowledge, validate, try non-verbal response, give benefit of the doubt, start with observation, not judgment, offer student choice to provide solution, etc.), but I realized that I hadn't given much conscious thought to how they applied to dealing with students, and it was helpful to have them made explicit.

As these guidelines may show, our wise colleagues encouraged us to detach ourselves from our annoyance, see our students with compassion, and use these sometimes-difficult moments to build stronger human relationships with our students.  To illustrate their wisdom, our colleagues told a few stories and showed us some video they had taken.

One wonderful story was this: when one of these colleagues was a student teacher, she went with her mentor teacher and their students on an overnight trip early in the year.  At a certain point late in the night, one student went missing.  The mentor teacher was annoyed and upset, and clearly, said my colleague, angry with the student who had slipped away.  When they finally found the student, my colleague expected her mentor teacher to light into the kid, or at least show that she was upset.  Instead, she told us, the master teacher opened her arms, gave the kid a huge hug, and said, "We were so worried about you!"  For my colleague, this was an eye-opening experience--and as she told the story, we were right there with her.  A good teacher doesn't let her students off the hook, but always shows that she cares about them.  If she doesn't, they won't learn.  According to my colleague, that kid, who was going through a difficult time, had a good relationship with her teachers for the rest of the year.

The video that our colleagues shared was also really interesting.  The clips we watched were only a minute or two each, but it was amazing how much was happening in those minutes.  I've never seen so clearly how a good teacher can have a dozen meaningful interactions, with a dozen different students, in the course of only a minute or two.  A look, a wave, a few words--each time this happened, a relationship was strengthened and a student was helped to, as my wife says, see his or her own thought process.  My colleagues were holding up a mirror--a compassionate mirror--to their students, and their students clearly felt supported, cared for, and encouraged.

Not brain surgeons, not instructors, but people!
These colleagues have also spent a lot of time working together on planning classroom activities (how to introduce books, how to facilitate discussions, structures for bringing out responses and helping students to organize their thinking, etc.), but very little of what they do is what I would call "instruction".  They are not brain surgeons, not instruction manuals, but people, helping other people.