Tuesday, May 21, 2019

Why does Kim Marshall promote the idea that schools can overcome poverty?

My department chair subscribes to the Marshall Memo, and I usually read all of it. I like the way it reminds me of what's important. This week, though, Kim Marshall included a piece called "Pushing Back on Eight Outmoded Beliefs." In the piece, originally published in the official publication of the Michigan Association of School Boards and written by Marshall himself, Marshall declares that it is "outmoded" to believe that "poverty is destiny."

In some sense, okay, poverty isn't destiny and never has been. Ben Franklin, Frederick Douglass and a lot of other heroes of mine have shown that some people can escape the fate that poverty usually points to. But to say that the idea is "outmoded" is pretty galling, not only because it implies that this idea is just a sort of trendy fashion, not only because there is no reason to think that poverty is any less determinative than it has ever been, but also because a more pervasive and pernicious myth is that schools CAN rectify the injustices of an increasingly unequal society.

My first impulse, on reading Kim Marshall's article, was to curse at my computer. But then I thought, well, maybe in fact progress has been made. It's been a few years since I looked into the question of whether schools can overcome poverty. Last I looked into it, I found that, No, in fact, schools cannot overcome poverty. But maybe that conclusion is "outmoded." According to Marshall, "some schools are turning this dynamic around and closing gaps; Education Trust’s website showcases a number of these beat-the-odds schools and what they are doing: https://edtrust.org/dispelling_the_myth/ .

So, okay, I went to the Education Trust's website. I was skeptical, but willing to look at it. If schools were succeeding at overcoming poverty, I wanted to see what they were doing.

Unfortunately, what I found when I looked into it was typical of what I found a few years ago, and typical of education discourse in general. It's the kind of thing that finally got me frustrated enough that I stopped paying much attention to education discourse and research: an absence of evidence that bears even the most cursory examination.

I will try to be brief. The Education Trust is an organization that seems to spend a lot of time and energy showing that schools can overcome poverty. Every year, the Education Trust gives "Dispelling the Myth" awards to schools whose "success dispels the damaging myth that schools can do very little to help students overcome the barriers of poverty and discrimination."

To be clear, this organization looks around the ENTIRE COUNTRY to find schools that overcome poverty.  So you might expect that the award-winning schools would have both high poverty, as measured in the standard way by the percentage of students whose family income qualifies them for Free or Reduced-Price Lunch, and also success, as measured by graduation rates and test scores and so on. I looked to see if this was true. But, mirabile dictu, I didn't actually ever even look to see if the award-winning schools had achieved any academic success, because as I started looking at the high schools that had won this award, I was shocked to find that the first two schools listed (there were only three overall) were serving student populations that actually had significantly LOWER POVERTY than the state or national average.

Let me say that again, because I still can't believe it myself: most of the schools that Kim Marshall and the Education Trust cites as showing that schools can overcome poverty actually have SIGNIFICANTLY LESS poverty than average to overcome.

I don't really know what to say about this, except that I hope Kim Marshall puts out a correction in his memo, and spends some time explicating the more important and very pernicious myth that schools can beat poverty. They can't. Schools can work with and nurture the students we are given, but the only real solutions to poverty lie outside the schoolhouse walls.

Notes about the particular schools:
The Education Trust lists only three schools. The first, Imperial High School in California, has 40% low-income students. Statewide 58% of students are low-income (meaning they qualify for Free or Reduced-Price Lunch). At the second, Jack Britt High School in North Carolina, 28% of the students are low-income. Statewide, that number is 58%. That's when I threw my hands up and started to write this blog post. But I did go back and look at the third, which though it's called "Pass Christian High School" seems to be a secular public high school in Missouri. Pass Christian actually does have more low-income students than the state average, if not by much (56% as compared to 50%). But still. If Education Trust combs the country looking foro schools that beat poverty, and this is what they come up with, I don't think we've proved that poverty can be overcome at the schoolwide level.

Wednesday, January 28, 2015

The best book I've read this year

The best two books I've read in the past several months were both historical fiction: first The Radetsky March, Joseph Roth's amazingly astringent 1932 novel about the decline of the Austro-Hungarian empire, recommended to me by a friend who's spending the year in Vienna; and now The Lions of Little Rock, a 2013 YA novel by Kristin Levine, recommended by my 10-year-old daughter.

My daughter had been trying to get me to read this book for weeks. Yesterday's snow day gave me a chance to start it--and once I started, I couldn't stop. The Lions of Little Rock, about the school integration struggle in Little Rock in the late 1950s, is maybe even better than the Roth. I was floored by its apparently effortless depth and wisdom. It's about a 7th grader, and it's packaged like a book for tweens, but everyone should read it.  It's a more gripping book than Warriors Don't Cry, and it's both better and less morally questionable than To Kill a Mockingbird. If my ninth grade students could get over their reluctance to read books aimed at younger children, I think they'd love it.

Again I am amazed at the quality of the YA books produced in our time. We are living in a golden age of children's literature, and we should thank our lucky stars.

Monday, January 19, 2015

Informing parents about missing student work raises GPA and raises test scores for Math--but not for English

I read an interesting study yesterday. Here's my take:

The Study
A New York Times article yesterday referred to an interesting study suggesting that getting more information to parents about their kids' education is low-hanging educational fruit.

According to the study's author, Peter Bergman,  a professor at Columbia, there are three basic problems in this area:

1) parents don't have high-quality information about their kids' grades and work completion

2) parental bias means that parents think their kids are doing more homework than they really are

3) the short-term bias of children (Bergman says they have "higher discount rates") leads them to conceal poor performance and poor homework completion from their parents

The literature about parental involvement has been ambiguous, according to Bergman, because it has been difficult to distinguish between cause and effect. Studies have even found negative correlations between parental monitoring/involvement and student achievement. This could either be because parental nagging is counterproductive or because parents tend to monitor and nag kids who are underperforming. Bergman thinks it's the latter.

Bergman models the situation using some fancy-looking game theory, but his experimental design is simple: the treatment group, at a California school serving mostly poor kids, got messages about missing assignments, including specific information like the date and the page number of the textbook. The control group got the default amount of information that school and teacher provided parents: report cards four times a semester and teacher contact as the teacher chose (usually very little).

Positive Results: GPA goes up; math test scores go up
The results were positive: in the treatment group, both overall GPA and math test scores increased by about .2 standard deviations. That's a pretty big increase.

This is not surprising, but it does point to an obvious lesson: schools and teachers should move to as transparent a grading system as possible. Parents should have continuous access to their children's assignments and grades. As a teacher, this will push me to be more timely and thorough about updating my gradebook, but that's a good thing. Or--it would be a good thing if I were a math teacher.

Unfortunately--as usual--English test scores don't change
Another interesting result of the study--for me, more interesting than its headline finding--was that the English test scores of the treatment group did not go up. (Well, they went up .04 standard deviations, but that's not much).

As Bergman notes, this disparity is typical--many studies find it easier to raise math scores than English scores (2). But it is nevertheless a real puzzle, one that calls into question the effectiveness of English class.

Informing parents about missing work seems to have had some direct effects. Parents who were informed about missing assignments were in better touch with the school, and more of them attended parent-teacher conferences. Parents in the treatment group took away twice as many privileges from their children (mainly access to screens and friends). Students in the treatment group skipped class less, completed more of their assignments, and attended after-school tutoring more often.

Therefore, this experiment found that a group of students who had more engaged parents, who went to class more often, who did more of their schoolwork, and who went for help after school, nevertheless did not make any more improvement on their English test scores.

This is a striking result.  Going to math class and doing your math homework really matters. Going to English class and doing your assigned English work seems to matter much less.

Results like this should make English teachers very uncomfortable.

How might English teachers respond?
An English teacher at this school could interpret these results in three ways:

1) the purpose of English class is not really to improve reading ability, so this result is irrelevant, and I am going to keep doing what I'm doing.

2) the purpose of English class is to improve reading, but coming to class and doing the assigned work does not seem to serve that purpose, so I am going to change my practice.

3) the purpose of English class is to improve reading, and although coming to class and doing the assigned work does not seem to serve that purpose very well, changing my practice would be inconvenient, difficult, and unnatural, so I am going to keep doing what I'm doing.

I think most English teachers respond, if unconsciously, in the third way. Here's why:

Education is conceptualized in a way that fits math, not English
English teachers work within a system that is set up for teaching subjects, not for getting kids to read a lot. The other academic subjects, including math, are about teaching discrete concepts and techniques, while English is about continual gradual improvement in the exact same activity students have been practicing since they were in first grade. In English there is no proper "content," and the usual ways of thinking about education are inappropriate and inadequate for thinking about reading.

The usual way of thinking about education is that the teacher teaches a skill or concept, and then students practice or conceptualize on their own and get feedback from the teacher about their performance. For me, this model was most vividly represented by the Australian researcher John Hattie, who in his book Visible Learning says that the "heart of the model of successful teaching and learning" is like rappelling in that "the goals are challenging, specific and visible," the skill being taught does not come naturally, the experience of learning is "exhilarating," and "it is abundantly clear what the success criteria are". This may be a good model for learning a single skill like rappelling off the top of a building, and it may work to some extent for subjects like math, science, history or foreign languages, in which there is "material" to be learned or "skills" to be mastered, but it is not a good model for improving your performance in an activity, like reading, that you have been doing over and over for many years.

The best way for students to get better at reading is probably to spend a lot of time reading interesting material, and to spend some time discussing the meaning of interesting or difficult texts. Unfortunately, neither the reading or the discussing fits easily into education's standard conceptual framework. So instead English teachers often assign work and plan classroom activities that are neither actual reading of texts nor discussion of texts, and it doesn't really matter if our students do the work or take part in the activities.

Bergman's study shows that we should be more transparent about missing student work, but it also shows that we English teachers should be rethinking our practice.

"Parent-Child Information Frictions and Human Capital Investment:
Evidence from a Field Experiment" by Peter Bergman

(2) This phenomenon is discussed somewhat in Bergman's paper, and in more detail in a 2010 paper by Eric Bettinger. It's also touched on in a 1996 meta-analysis of summer learning loss by Cooper et al. The paper about summer learning loss is interesting, since it finds that although on average the summer learning loss is greater in math, for poor kids it is greater in reading. Food for thought.

Tuesday, May 20, 2014

The new Common Sense Media report on reading

(I have been taking a year off from blogging while I think things through on (maybe) a deeper level, but I wanted to record a quick take on a new paper on reading that's been in the news.)

A new report by Common Sense Media reviewing the data on how much kids read has been getting some press recently. In the Times, Frank Bruni used it as the jumping off point for a column celebrating reading. My department chair sent us a link to the report, and asked what we thought. Here are my quick thoughts.

The standard takeaway from the report seems to be that kids are reading less now than they were 20 years ago. My department chair used the phrase "hell in a handbasket," and Bruni calls the change "marked and depressing." I'm not sure it's as bad as all that. But if the data the report discusses is somewhat open to interpretation, it does seem clear that our society can do better at helping our schoolchildren read. Because reading seems clearly to be the most important academic skill, this matters a lot.

The report's first two "key findings" are what have gotten the most attention, and they also seem most important to me. The report's other key findings are either of less general interest (gender gaps, racial/ethnic gaps, impact of e-readers, etc.) or kind of obvious (kids whose parents read aloud to them and have books in the house are better readers). The first two key findings are:

1) all kids, but especially older kids, are reading less for fun than they used to;

2) while reading comprehension scores for elementary-age kids are higher than they used to be, reading comprehension scores for older kids have not improved.

What's interesting about these two findings, if they're accurate, is that there are two apparent contradictions: kids in the younger grades are better at reading than they used to be, but kids in the older grades are not; and the reduction in pleasure reading is not translating to a decrease in reading scores.

Two possible (but I think unlikely) interpretations:

One way to interpret these findings is to think that reading for pleasure doesn't matter much. That's possible, but unlikely, given the high correlation between pleasure reading and reading scores.

Another possibility is that the findings are not accurate, that older kids are reading just about as much as they used to, but that some of their reading is not showing up in the data. As far as I can tell, there is some possibility that some of what teenagers are doing on their screens is not being counted as reading, even though it is practically the same as stuff that in the past, when it wasn't done via a screen, used to be counted as reading. A lot of what my son does on his iPod touch is reading about professional tennis and rock climbing. When I was his age, I spent hours every week reading bicycling magazines. Reading an online article or blog post and reading an article in a print magazine are basically the same activity, but I looked like I was reading, and he looks like he's on facebook.

My take:

While these studies may not be catching all the reading that's really happening, I think they're probably mostly accurate. Older kids probably are reading less. Some of this may be due to the improvement of competing non-reading activities like youtube, instagram and facebook (technologies that can be seen as improved versions of the telephone and the television). But this wouldn't explain why younger kids are getting higher reading scores now, so I think there is something else going on, too.

What I think has happened is that we have put a lot of energy into trying to teach, through direct instruction, the "skills" and "strategies" of reading. There has been a huge push, at all levels, to turn reading instruction into a mechanical process that can be broken down and implemented piece by piece. What there has not been is a huge push to encourage reading as a pleasurable and valuable activity that people might want to engage in on their own, either for entertainment or (like my son's reading about tennis and rock climbing) for information about the world.

The increase in direct instruction in skills and strategies may have boosted reading scores on the NAEP in the elementary school years, but they have not made 17 year olds better readers. We seem to be having short term success, but the short term success is not translating into long term success.  This is a pattern that I noticed last year in data about the reading scores of students in Waldorf schools. Waldorf education doesn't teach reading at all until kids are 7 or 8. As you might expect, Waldorf students score poorly on reading tests in early elementary school. By eighth grade, however, they have caught up. By the end of high school, they may be pulling ahead.

It also strikes me that the direct instruction that may produce higher scores in elementary school could be having a counterproductive effect in the long term. It's very possible that the way reading is now taught discourages reading for pleasure.

So if it were up to me, I would try to spend more energy on encouraging pleasure reading, especially at the high school level, when pleasure reading drops off precipitously.

The report: https://www.commonsensemedia.org/file/csm-childrenteensandreading-2014pdf-0/download

The Times column: http://www.nytimes.com/2014/05/13/opinion/bruni-read-kids-read.html

Sunday, September 22, 2013

Fancy math in educational research

I was reading an economics blogger who sometimes writes about inequality, and a post of his (http://owenzidar.wordpress.com/2012/12/29/what-would-happen-if-wed-greatly-expand-charter-school-availability/)sent me to a paper by an economics grad student at MIT about charter schools. The paper (http://economics.mit.edu/files/8429) seemed competent--the author is handier with his math than I am--but basically just marginal embroidery. The paper is another angle on the inadequacy of educational research. Basically, the lesson I take away is that fancy math doesn't matter--what you need is more and better data.  

First, I haven't read that many papers about education that attempt such elaborate mathematical modeling as this one.  Here is a screen shot of one of the pages:

Second, it  was remarkable that such elaborate modeling allowed the author to achieve virtually no new insight. The paper begins by acknowledging that there are two ways to understand the success of Boston charter schools: either the success is due to the non-union faculty and "No Excuses" practices of Boston charters, or the success is due to the self-selected student population. Yes, true--and the author's extremely elaborate model only allows him to conclude that if charter schools make a big difference in the achievement of poorer students, then it will be important to figure out how to get more poor students into charters.

This is an entirely obvious conclusion that you might have reached without doing any of the fancy modeling and without spending years in graduate school at MIT. The key issue here is whether it is the schools or the non-school factors that lead to the higher test scores, and this paper sheds no light on that issue.

Economists often write about education, because of the apparent strengths of both the economists (they're arguably better at math than ed school Ph.D.s) and the education data (it's arguably more abundant and rich than economics data). But this paper shows the limitations of both of these apparent strengths: the economist's fancy math is deployed to no effective purpose; and the education data is clearly inadequate.

If we want to know whether the Boston charter schools are markedly more effective than the rest of the Boston system, we would probably have to conduct an actual scientific experiment: randomly assign a large number of students, not allow the charters to drop their lower-performing students, and see what happened. Until then, I will continue to assume what I have never seen disproven, that schools alone cannot overcome achievement gaps, and that the higher test scores of students in Boston charters does not mean that eliminating teachers' unions and adopting "No Excuses" policies would help poor kids learn more.

Tuesday, August 6, 2013

The Death of Theory in Education?

In literary studies, the age of "theory" was coming to its end when I was in college, twenty-some years ago.  I took a class or two in which we read Terry Eagleton's little primer on literary theory and sampled work by a various people who'd been read a lot in the seventies and eighties (Derrida, etc.), but as I recall, my friends and I thought the more theoretical stuff was pretty much a joke. Today I read a piece by an economist making the case that the decline of theory has been a trend in economics, too, and indeed across all the disciplines.  Physicists have hit a theory wall; empirical sciences like biology are ascendant; published economics papers are down from over 50% theoretical in the 60s, 70s and 80s to less than 20% theoretical in 2011; and math is increasingly reliant on proving things by means of brute computing force rather than theoretical deduction.

Why has theory declined?
That last example points to one obvious reason for the shift from a theoretical approach to an empirical approach. Computers allow for data-crunching and simulation-testing that would have been unthinkable a few decades ago. But computers are only one possible factor. The economist whose post I read suggests that the waning of theory might be due not only to competition from the now-stronger alternative of empiricism, but because theory itself has been played out (he expresses it in econ terms: "Maybe we've temporarily exhausted the surplus gains available from ramping up our investment in theory.")

As I see it, computing is certainly a huge part of the decline of theory, and the endogenous "exhaustion" of theory itself may be another. I wonder if another factor could be the fall of the Soviet Union. For over a hundred years before the time I was in college, the world had a very vigorous non-stop debate over how to see the world. Then, with the rise of the USSR, competition between the two hegemonic political systems fueled theoretical debates even in fields very far removed from politics (literary studies, for one). When I was in college, the global political competition abruptly ended, and the theoretical debates started to taper off as well. The decline was particularly steep in economics, but it happened in lots of other fields, too.

Connection to Education?
Is this related to education?  I think so.  It's certainly true that theory is less important in education than it used to be. This is true in the literature, but we can see it all around us. When I taught in Harlem in the mid-nineties, the headmaster of my school was an old poet who talked about Paulo Freire all the time. By the time I left, a few years later, he'd been forced out by his board and been replaced by a young corporate executive. Twenty-five years ago, Peter McClaren came out with Life in Schools, my copy of which cites the author's "cutting-edge theoretical work"; last year, when I first read something McClaren had written, my first instinct was to make fun, not only of his rock-star style choices, but also of his Marxism.

So the decline of theory in education does seem related to the triumph of capitalism. Recently I've been thinking about how education is both a public good and a private good. Over the past twenty-five years, the standard view of education has moved away from seeing it as a public good and has come to see it more and more in individualistic terms. There has also, of course, been a push for a more "free-market" system in education.  Both of these shifts, I think, would have been very different if the USSR were still around--not because the USSR was doing education in particular so differently, nor because Diane Ravitch or whoever would be at all sympathetic to the communist system, but because theoretical debate in general would be so much more vigorous. As it is now, we find it difficult to believe in theory at all anymore.  I remember reading Flatland in high school and being really struck by the idea that in a two-dimensional world you just can't imagine a third dimension.

What will happen? 
We will eventually rebuild our theoretical discourses. But it might take a while. In the meantime, while we are stuck with capitalism and with appeals to "evidence-based" approaches, we need to be wary of going with the flow.

What should we do?
I like liberation theology's idea that we should have a "preferential option for the poor"; in the current era, maybe we should have a preferential option for theory and against the rich and powerful. That is, maybe we should:

  1. Look very critically at the evidence, and remember (recalling the Foucault we read lo those many years ago) that when we claim our approach is based on evidence it is probably based on assumptions that have nothing to do with evidence, and 
  2. Look very critically at anything that is supported by rich people or would serve to benefit rich people.
Other ideas?

Thursday, August 1, 2013

ETS calls for Graduate Schools of Educational Public Health and more attention to poverty

ETS (!) has sponsored an excellent report on poverty and education.  Written by Richard Coley of ETS and Bruce Baker of Rutgers (and schoolfinance101), the report says what should be obvious to any observer and what many other excellent reports have said over the past forty years: that poverty is a problem for education, but education alone is not a solution to poverty.  Most of the report does the noble work of making (again) a detailed case that there is way too much poverty in the US, that we don't do enough about it, and that this poverty has serious negative educational consequences. The whole report is worthy, but I noticed a couple of things in particular:

A call for a focus on educational public health
The report's executive summary contains a wonderful sentence expressing a sentiment I wish we saw more often:

Given the strong connection between educational success and economic disadvantage, we might
expect education policy to focus on ways to overcome the effects of poverty on children.

In other words, they want schools of Educational Public Health!

What to do
Despite the report's clear implication that fixing poverty would be a good way to improve educational outcomes, the authors unfortunately choose, apparently from modesty, not to recommend any non-educational measures: "There are other strategies that fall outside of the education arena — tax policy, job creation, minimum wage policy, etc. — that also are outside of the purview of this report." The report does, however, make seven specific suggestions for actions within the "education arena", all of which ought to be common sense:

Increasing awareness of the incidence of poverty and its consequences. Child poverty costs the United States hundreds of billions of dollars per year. Current poverty levels, combined with the growingwealth gap between those at the top and bottom of the distribution, threaten to destabilize our democracy and limit the upward mobility of children of future generations.
Equitably and adequately funding our schools. The economic downturn has taken a toll on state school funding and on targeted programs like preschool that can help disadvantaged children. There is a need for better coordination of federal and state education programs targeted at poverty.
Broadening access to high-quality preschool education. High-quality early childhood education programs improve the educational outcomes of all children, but particularly for low-income children. The administration’s proposed major expansion of preschool programs across the country should be supported.
Reducing segregation and isolation. Many of the nation’s schools are increasingly segregated by race/ethnicity and income. Each student should have the opportunity to attend schools with peers from diverse social and economic backgrounds.
Adopting effective school practices. School policies that have been documented by research and practice to be effective should be broadly adopted. Examples include class size reduction, longer school days and years, and tutoring.
Recognizing the importance of a high-quality teacher workforce. Attracting and keeping high-quality teachers in high-poverty classrooms should be of the utmost priority and may require special incentives. 
Improving the measurement of poverty. The poverty rate is an important social and economic indicator that is used to allocate resources for scores of federal, state, and local programs. Work should continue to expand the official definition of income to include government spending directed at low-income families and to recognize cost-of-living differences across regions.

These aren't new ideas, but as the poet says, it is necessary to write about the same old things
in the same way, repeating the same things over and over. And who knows, maybe this report will get a lot of attention, and we'll stop spending so much time designing ways for schools to compete for the richer students (in Hirschman's terms, we need a system where change is based on voice, not exit).  And, yes, we need a lot less poverty.

Tuesday, July 30, 2013

Atul Gawande's recent article is relevant to education, too!

There is an excellent Atul Gawande piece in the New Yorker this week about improving medical practice.  There are two main ideas in the article, both of which apply beautifully to education:

1) consistent execution of simple, straightforward ideas is more important than fancy new technology
2) to achieve consistent execution of simple, straightforward ideas we should use one-to-one coaching--in Gawande's words, "people talking to people."

Every teacher and education wonk in the country should read Gawande's whole article, which is, like his other work (checklists! standardizing good practices!), clever, important, and very readable.

Consistent execution of simple, straightforward ideas
A lot of us teachers--me included--worry a lot about what the right curriculum is, think a lot about the big picture of lesson planning, argue about, say, whether kids should be assigned reading or should pick it themselves, and spend hours and hours on new technologies.  These are important things to worry about, and we should certainly be figuring out what matters for our students' learning, but what is probably much more important is our consistent and effective execution of what we already know matters.

Gawande's article begins by contrasting the immediate adoption of anaesthetic use during surgery with the many decades it took for antiseptic measures to take hold, even though the antiseptic measures were known to be hugely important and could have saved many, many lives.  One reason for the quicker adoption of anaesthetics is obvious: anaesthetic, unlike antiseptic measures, has very immediate and direct benefits that are obvious during surgery.  Surgery is painful and difficult without anaesthetics, and it's pretty easy with them.  Easy, but not necessarily effective.  To be effective, the surgery has to be antiseptic--the doctor has to wash his hands and instruments, has to wear freshly cleaned/sterilized clothing and mask, and has to soak up the blood and fluids with fresh gauze instead of the sea sponges that doctors used to use over and over, never sterilizing them. According to Gawande, these antispetic measures didn't catch on not only because they didn't have immediately observable benefits in combatting a visible and obvious problem, but because while anaesthesia made life a lot easier for doctors (imagine cutting into someone who's screaming and writhing in pain), the antiseptic measures actually made life considerably harder for them (Gawande describes early aseptic surgery as requiring surgeons "to work in a shower of carbolic acid").

This distinction between what is easy for the patient and what is easy for the doctor has its analogy in education.  Often, what is good for the student is not particularly easy for the teacher.  And often what comes naturally to the teacher is not what is best for the student.  It's hard for teachers to see this, and we need people to help us.  And even when we know what's important, we don't always manage to do it.  Again, we need people to help us. But what we need is not what we get.

One-to-one Coaching, not evaluation
If we know what teachers must do, but they (we!) don't always do it, how can we help them (us!) improve? In education, too often the only feedback teachers get is through an annual evaluation by a supervisor. These evaluations can be helpful, but we need much, much more.

I have a really great department chair.  She is thoughtful, hard-working, funny and wise, and she has gotten better and better at her job over the past eight years. I always learn a lot from my biannual evaluation.  But it is not nearly enough. I was evaluated this year, and Mary came to my class three times for a total of about an hour.  An hour every two years is just not enough observation. It's great, but it's not enough.

The other problem with our current scheme is that it combines coaching with evaluation. Having any observer in the room will probably change your teaching in some way, but having an observer who is going to be writing a report that goes into your personnel file and could theoretically be used to fire you is definitely going to change the teacher's practice--and the conversations afterward cannot possibly be as free and open with a supervisor as they might be with someone who was only a personal coach. The obvious answer is to have peer coaches who would not judge or evaluate, but only help.  This is essentially what Gawande describes in his article, and it is the obvious way forward for improving teaching--a much more humane and effective alternative to the methods pushed by ed reformers.

This is what teachers' unions should be working on. Instead of abolishing teacher tenure, peer coaching. Instead of a revamped evaluation system, peer coaching. Instead of VAM evaluations, peer coaching. Unions should be leading the way in designing non-punitive teacher-improvement programs.   Of course this is happening to some extent (for instance, one of the wise consultants Leafstrewn has had helping with its work on reading in the content areas is, I think, an expert in peer coaching), but we need much, much more of it. 

How it would work
A peer coaching system would take time. I've done some observing of other teachers, and I've been observed by my colleagues, and the problems have always been twofold: (1) the observations aren't targeted enough; (2) the observations and conversations haven't continued long enough for a comfortable working relationship to develop and for problems to surface, be discussed, and be worked out. Gawande's article addresses both of these problems.

Gawande describes programs that are aimed at specific issues: teaching cholera patientsto treat   themselves with a simple rehydrating solution (a treatment that is actually more effective than the intravenous rehydration that, though the high-tech standard, is impracticable in many places); and teaching obstetric nurses to make sure to execute the couple of dozen practices most important for the health of the mother and child (washing hands, encouraging skin-to-skin contact, monitoring the baby's temperature, etc.), practices that if followed could save millions of lives each year. These programs are staffed by coaches who, though trained, are neither powerful nor very experienced, and the coaching is not compulsory (the nurses don't have to take the advice if they don't think it's helpful). The key is the personal relationship and trust that develops over time.

Gawande describes a young nurse who observes and coaches an older, more experienced obstetric nurse.  After many visits, the older nurse started to change her practice. Gawande asks the older nurse why she listened to the younger, less experienced one; in the beginning, the older nurse said, she didn't.

"The first day she came, I felt the workload on my head was increasing." From the second time, however, the nurse began feeling better about the visits. She even began looking forward to them.
    "Why," I asked.
     All the nurse ccould think to say was "She was nice."
     "She was nice?"
     "She smiled a lot."
     "That was it?"
     "It wasn't like talking to someone who was trying to find mistakes," she said. "It was like talking to a friend."

We need to observe each other more, and not just for one or two times, but repeatedly over long periods, and like these nurses, we need to focus on the basics. I don't think teaching is quite as amenable as doctoring to standardization (for schools to use The Cheesecake Factory as a model, as Gawande wants hospitals to do, would be a mistake), but some questions are relevant for all English classrooms: what is the ratio of teacher talk to student talk? how much are the students reading? who is paying attention and who is not? where are the questions and ideas coming from? are our instructions clear? how much time are we spending/wasting on transitions and instructions? 

Teachers can certainly work on these basic issues on their own, and some of these questions are amenable to checklists or in-class assessments, but the same is true of the fundamentals of obstetric nursing. As Gawande's article shows, and as I know from long experience, just because we know what we should do doesn't mean that we actually do it. To learn to break our bad habits we need lots of sympathetic, non-judgmental help--and not just in the office around lesson-planning, but in the classroom, around execution. So when the current wave off top-down, authoritarian ed reform finally subsides, I hope we will see a boom in peer-to-peer coaching.

In any case, I recommend the whole Gawande article.

Why do we need MOOCs when we have books?

Over the past few years I've gotten interested in economics, and this summer I decided to finally go through an introductory econ textbook.  It's a good way to learn.  I go at my own pace, I can re-read the parts I don't understand, and I can do, or not do, as many of the problems at the end of each chapter as I think I will be helpful. Would taking an actual econ 101 course be better?  Maybe, but maybe not.

Learning economics in this way makes me wonder, of course, about school. I think about the high school courses I teach, and I think, as usual, that we should try to have a lot of our students' learning be done through independent reading. But I also think about online college courses--MOOCs, to use the appropriately silly acronym. And I wonder--what's the point of a MOOC, when we already have books?

Of course, you might ask, what's the point of lectures, when we already have books.  Lectures, as the name implies, started before there were printed books: the lecturer read his book/lecture notes, and the students copied down what he said, so that they could read and re-read it later, at their own pace.  After Gutenberg, lectures weren't as important, but they continued for various reasons (people like watching people, there could be personal contact with the lecturer at other times, etc.). But as many college students have noticed, lectures are not a very good way of teaching.  I skipped at least half of my lectures in college, and I did okay; I had the textbook.

So why MOOCs? And why now? 
If watching someone like Michael Sandel is so great, we could already, before the internet, watch him on videotape or film.  Why didn't community colleges in the seventies and eighties just show films, instead of hiring actual professors? Why are MOOCs happening now?

To answer that question would require going into a whole array of cultural shifts, but it's worth pointing out that places like Harvard and MIT are not, I don't think, going to outsource their teaching to virtual professors anytime soon. MOOCs, like so many other "disruptive" educational innovations, are a matter of providing a cheaper and shoddier product to the middle and lower classes, while providing ever more artisanal craftsmanship to the upper classes.

Wednesday, July 17, 2013

Moving to a free market in education could lead to much higher costs

In between dealing with personal business this summer I've been teaching myself economics and reading a bit in the socioloy of education, and a news story today connects to both.

Health Care and Education: competition in excluding people
There are a lot of similarities between health care and education. One significant similarity is that some people are much more difficult and expensive than other people to educate or keep healthy, so there are huge economic incentives to simply not serve those people. In health care, what this means is that private health insurance companies have often been essentially competing with each other, not to get customers by providing better service, but to not get the customers who are unhealthy--exactly those who most need it.  The same is true in education.  The children who most need a high quality school are exactly the children that schools in a market system would be doing their damnedest to exclude, and that many if not most charter schools are excluding (Bruce Baker, at schoolfinance101, just wrote a piece about this phenomenon in some of Newark's most highly touted charters, which are dropping young black men from their grade cohorts at amazing rates--up to 75%). 

The economic truth of this dynamic--that there are huge costs and inefficiencies in a market system that allows competition in excluding people from your services--can be seen really clearly in health care. The dynamic is clearer in health care than in education, because in health care the US has for many years had a "market-based" health insurance system.  As compared to other developed countries, all of which have some form of government-imposed universal health care, the US spends close to twice as much on health care, with worse results. This is an amazing argument for socialized medicine, and by extension for socialized education. 

Universal = Inexpensive
The same dynamic can be seen in the remarkable news, reported in today's Times, that health insurance premiums in New York are, now that Obamacare has required everyone to get health insurance, going down by at least 50%.  New York State used to have a system in which health insurers were required to take anyone, but not everyone was required to buy health insurance.  This is in some ways analogous to a school "choice" system in which not everyone has to go to the local public schools.  What happens is that the most able students get siphoned away by charters, private schools and other districts, and the core local schools are left with the most expensive and difficult cases.

The Ed Reformers want to move from a more efficient, more socialized system, to a less efficient, more expensive "market" system.  What will happen is the reverse of what is happening under Obamacare.  We will go from having, like every other developed country in the world, an education system that offers reasonable and safe schooling for all, to having an education system that is as dysfunctional as the US health care system has been: not only completely excluding 15% of the population, but costing more for everybody else.

Horace Mann knew this a long time ago Here's what the wise Horace Mann, Nathaniel Hawthorne's brother-in-law, said over a hundred and fifty years ago, when he was the Secretary of the Board of Education in the great state of which Leafstrewn is just a muddy puddle: 

"Now surely nothing but universal education can counterwork this tendency to the domination of capital and the servility of labor... Education, then, beyond all other devices of human origin, is a great equalizer of the conditions of men,--the balance wheel of the social machinery."

Horace Mann's words raise a number of questions about the purposes and effects of education. I'll write about these questions a bit over the next few weeks.

Monday, July 1, 2013

What might the new NAEP scores mean?

The "Nation's Report Card," otherwise known as the report on the most recent National Assessment of Educational Progress, or NAEP (I always thought it rhymed with "Jeep," but it may rhyme with "tape"), has just come out.  The NAEP is widely considered a good test, and since the NAEP is not high-stakes, there is little reason for schools or students to cheat, or for schools to attempt to teach directly to it.  I remember Mike Dukakis telling an auditorium full of Leafstrewn students that NCLB was unnecessary because we already had the NAEP.

The NAEP is a good test, and because it is such a good test, and because reading and math abilities change so extraordinarily slowly, there aren't the kinds of variations in the NAEP that there are in, say, the MCAS.  MCAS scores have gone up dramatically over the past fifteen years, while the NAEP scores are always more or less the same.  Therefore, the release of the NAEP scores is a little like a Rorschach  test.

Optimists who favor ed reform will point to little fluctuations and say, See, NAEP scores have risen over the past four years!  People who are against ed reform will say, Look, 17-year olds reading scores are below where they were in the nineties! Nerds will rightly point out that the demographics of the students taking the test have changed significantly over the years, and that breaking out subgroups can be interesting (for instance, black kids' scores have improved much more than those of white kids).  Contrarians might say, the scores haven't changed much, so school doesn't matter.

I'd like to point out an interesting feature of the new report and suggest a possible explanation for it.  I'm not sure I'm right, but my explanation goes along with some of what I have said in the past about the difference between short-term thinking and long-term thinking.

NAEP scores rise for 9 and 13 year-olds
The new "Nation's Report Card" has a very clear lede, and here it is:

Both 9- and 13-year-olds scored higher in reading and mathematics in 2012 than students their age in the early 1970s . Scores were 8 to 25 points higher in 2012 than in the first assessment year. Seventeen-year-olds, however, did not show similar gains. Average reading and mathematics scores in 2012 for 17-year-olds were not significantly different from scores in the first assessment year.

In other words, the most significant result coming out of this year's data is that over the past forty years, scores of nine and thirteen-year olds have gone up somewhat, but scores of seventeen-year olds are not significantly different than they were forty years ago.

One way to interpret this is to say, Well, we sure are doing a better job in the elementary schools; but high schools just aren't getting better.

This view--that elementary schools are improving but high schools are not--doesn't seem totally unreasonable, but there's an interesting problem with it that I have not seen anyone point out. The problem is that this result doesn't fit well with the standard theories of education, which would expect increased achievement by K-8 students to lead directly to increased achievement by 17-year-olds. The standard theories of education--that is, the views of people like Tim Shanahan, the ed reformers, and so on--see learning as essentially a step-by-step process of learning skills. According to this view, it is important to teach reading early, and to focus heavily on skills.  But if you have this view, then the higher scores of 9 and 13 year-olds would predict higher scores of 17-year-olds, and so the lower scores of 17-year-olds would seem to imply, not only that high schools were not getting any better, but that they were actually getting significantly worse.  For you would expect that if you took two 13-year-olds and put them through the same secondary education, the one who was a better reader as a 13-year-old would end up a better reader at 17. For a much better reader at 13 to end up the same as his less-skilled peer at 17 would seem to imply a much worse secondary education.

Have US high schools gotten worse over the past 40 years?
The NAEP scores are, if you follow the standard model, evidence that high schools have actually gotten worse over the past forty years.  This is certainly possible. It's also possible that there are demographic issues involved (changes in dropout rates could affect the scores of 17-year-olds).  But there is another possibility: it could be that the short-term thinking that has been increasingly prevalent over the past few decades has actually led to short-term success, but to a kind of short-term success that has not supported long-term improvement.

I can see two ways this could work.  One is a direct cost: the short-term teaching could be actively bad in the long run.  For instance, it might turn kids off to learning or reading.  If you drill kids for tests, the drilling might improve their scores but make them less creative thinkers.  Two, there might be an opportunity cost: by teaching skills or teaching to the test, you might do less of the kinds of things that prepare kids for learning later on.  For instance, you might read aloud to the kids less, or you might cut down on recess, or you might reduce the time allotted for free, creative play.  Any of these could be imagined to result in lower reading scores a decade later.

Evidence that this kind of short-term/long-term trade-off might be possible can be found in the studies on Waldorf schools that I wrote about a couple of weeks ago.  Waldorf schools do not do any explicit teaching of reading skills until the second grade, and their reading scores are, not surprisingly, markedly below those of other schools in the early grades:

The Waldorf schools do, however, catch up.  I'd love to see data on where those students are at the age of 17.  It seems very possible that by the age of 17 the Waldorf kids, who while everyone else was drilling on phonics instead did a lot of listening to stories, singing songs, reciting poems, and observing nature, might be way ahead.  And that may be the same dynamic we see in the NAEP scores.

If our goals are long-term, why are we all thinking short-term?
As I have repeatedly argued, getting better at reading is a very long-term process, and yet far too much of our thinking as teachers, like far too much of the discourse about education, focuses on the short term.  Most discussion of lesson-planning, for instance, seems based on teaching students a discrete skill that they have never attempted before.  John Hattie, in the introduction to his magnum opus, Visible Learning, offers my favorite example of this short-term thinking: Hattie describes in loving detail the excitement of an initial lesson in rappelling down a building, and then says that this is "the heart of the model of successful teaching and learning."  This is absurd, since reading is neither dangerous nor novel to most of our students, and Hattie would find that teaching rappelling to people who'd already been rappelling for ten years would be very, very different. But Hattie's absurdity is just an extreme version of the kind of thinking we all do.  We are after all called "teachers," and just as it is natural to think to ourselves, "So, what particular skill am I going to teach today?", it is also natural to want to measure students' improvement over the relatively short time periods of a unit, a semester, or a year.  So, under MCAS and NCLB, we now give students high-stakes tests every year, and we are moving to a system by which teachers are evaluated by the results of these short-term assessments.

I doubt this is wise, but in the end it may not do too much harm. The remarkable stability of the NAEP scores is a healthy reminder that changes in educational regimes in the US have not made much difference to test scores. On the other hand, the NAEP may also mean that short-term thinking is ineffective: it may lead to short-term successes, but those short-term successes do not necessarily lead to longer-term success.

Monday, June 24, 2013

Results of a survey on independent reading

I asked one of my ninth grade classes to write my a quick note about how independent reading went for them this year. I asked specifically about how they felt about independent reading, how it had changed, if at all, over the course of the year, and what they foresaw for the future. A few kids were absent; I got thirteen responses.  Two were weakly positive; two were strongly negative, and nine were strongly positive.

I expected to get more responses like the two that were weakly positive, saying that reading "wasn't bad" but not granting that it was actually good.  Instead, most students seemed pretty enthusiastic.  Maybe they were just saying what I was obviously hoping they would say, but I like to think they were really pretty happy to get an English class that gave them a lot of time and encouragement to read what they were interested in.  The strongly negative responses were perhaps the most interesting.  One of the kids read quite a bit (a couple dozen books), and the other read very little (four or five books).  But both predict they will never read voluntarily.  This is probably true.  The tone of these comments is a bit defiant ("Reading didn't actually change my year in any way whatsoever"; "independent reading this year sucked.  I hated it."), but I think they were pleased to get a chance to vent their feelings about having to do something every day that they feel they're really bad at.

In the end, I think the independent reading was a good idea.  I have some ideas about how to change it in the future, but I still think this is a direction we should all be taking.

Here are the responses, unedited except for redacting the name of one elementary school.

Weakly positive:

  • It wasn't bad.  I liked it better than class work.  If the book was good, I liked it.  It got better over the year.  I'll probably read more.
  • I think that the independent reading this year wasn't that bad.  It helped me read a lot, which increased my vocabulary.  I have read more this year than last year.
Strongly negative:
  • My independent reading has greatly increased.  I still don't like reading, but if I have to then I will.  Reading didn't actually change my year in any way whatsoever.  I most likely will never read again unless I have to.
  • I thought the independent reading this year sucked.  I hated it.  I never liked to read, and I never will.  the only time I would like to read is if I find a book that I actually like that doesn't happen often.  I Hate independent reading and I think you shouldn't do it next year.
Strongly positive:
  • My independent reading got a lot better for me this year as far as how much reading i did.  Because of the amount of reading we did this year i will continue to read a lot and enjoy reading.
  • I really started to like reading again.  This really going to help me in the future because I want to actually pick up a book and read compared to last year when I would spark note everything.
  • By reading a lot this year, it made me read more books that I really liked.  I also read a lot at home.  I definatly became a better reader and really enjoyed the unit. I think that I will read a lot this summer and next year.  I haven't read more in my life than I did this year.
  • Independent reading this year was at least for me an improvement over the past couple of years. At __________ our teachers never forced us to read that much because we would always be reading a book as a class.  The only problem was I didn't read that much compared to the others in our class.  I would like to keep improving on the amount that I read because my family do so much of it.  This was still a very good year for me even if it may not look like much it's probably the most that I've ever read in my entire life.
  • I liked independent reading a lot more than assigned reading. It gave me an option to find a book I liked rather than a boring book. I didn't read many books over the course of the year, But I read more than I did last year. I would like to read more in the future.
  • I thought that the independent reading this year was a great idea.  I hate reading as a class because I read slower so I have trouble staying up to date.  It has made me more enthusiastic about reading.  I think I will definately do more independent reading this summer.  At first I wasn't thrilled with the idea of reading on my own but as the year progressed I began to enjoy it.  Thank you for encouraging me to read more and thus making me enjoy it more. Definately do this next year.
  • Independent reading this year has helped me advance to a faster reader.  I've read 3 books the whole way through this year, and I haven't done that before.  I've challenged myself with more challenging books this year and I've also figured out what genre I'm most interested in.
  • Independent reading really opened my eyes to reading more.  Some books I really liked and some books I hated.  Independent reading in class really improved my reading skills.  I think you should continue to allow your students to do independent reading.
  • I thought independent reading was great for me.  Whenever I have to read, I really feel like I am reading for myself.  Reading did change for me.  I have developed my interest and skills in reading. In the future, reading will be closer to my fashion.

Tuesday, June 18, 2013

NOLA schools "improvement": Ed Reform is likely less important than time, funding and demographics

The NYT has a post up about Ed reform in New Orleans that's getting a lot of attention.  The post basically says that ed reform is helping lift student achievement but may be hurting communities in other ways.  The part about hurting communities is interesting, but the part that attributes increasing student achievement to Ed Reform seems dubious to me.

Bruce Baker of Rutgers (and schoolfinance101) has been pointing out for years that all charter successes are completely explicable by looking at two factors: (a) increased spending and in-school time (as at HCZ etc.); (b) different populations.  If you spend more money and time on a school, and if your student population has fewer poor, ELL or special ed students, then your test scores will be higher.  Baker has repeatedly shown that highly touted charters are actually spending more money and/or enrolling more advantaged populations.

The New Orleans schools have both increased spending (as the NYT article mentions but passes over quickly) and a distinctly different population (as the article does not mention at all, as far as I could see.)  This second point is important. The population of New Orleans is distinctly different post-Katrina (as the NYT reported).  It is smaller overall, it is more white and less black, and it is less poor.

My guess is that these factors are easily enough to explain the higher test scores.

Monday, June 17, 2013

Waldorf Schools are an interesting natural experiment showing that explicit reading instruction in the very early grades may be unnecessary

We have some family friends whose kids used to go to a (private) Waldorf school nearby.  The kids were wonderful, but I didn't know much about the school or its methods.  A few days ago, when I was saying something about whether children needed to read "informational text" in the very early grades, my wife said, "That's silly. Waldorf schools don't teach ANY reading until second grade, and those kids end up just fine."

I looked this up, and, as usual, my wife was right: Waldorf schools generally don't teach reading until second grade, use a whole language approach and avoid much explicit strategy instruction when they do teach it, and their students apparently end up reading just fine.  This is an important result, because it would seem to show that explicit reading instruction in Kindergarten and first grade may not be necessary, and that students certainly don't need to read much informational text to themselves in kindergarten and first grade in order to learn to read well later on.

Recent Studies
Until recently, most Waldorf schools were private, so skeptics could argue that if Waldorf students ended up being good readers, the students and families at those schools were distinctly different from the norm, so no comparison was possible.  Over the past couple of decades, however, a number of public Waldorf schools have opened, most of them in California, and two recent studies in the U.S.(Oberman 2007(pdf); Larrison et al. 2012) compare the results at these schools with those at traditional schools with comparable student demographics. The two studies find the same result: when it comes to reading on their own, students in the early grades in Waldorf schools are dramatically worse than their peers in regular schools, but by the later grades, the Waldorf students have caught up or surpassed the regular-school students.

The graphic below shows some of the results obtained in the 2012 study. The scores of the Waldorf students start well below average, then catch up by fourth grade, then seem to pull ahead.

These results are striking. When the same researchers looked only at the California Waldorf schools, so as to avoid issues with cross-state comparisons, the same pattern was seen, though with less dramatic divergence in the upper grades. When Oberman did a similar comparison on a more limited scale and with data from two years earlier, she found a somewhat similar pattern--Waldorf students starting out behind and catching up, if not pulling ahead.  A New Zealand study comes to the same conclusion: Waldorf students do badly on reading tests when they are 6 and 7, but by the time they are entering adolescence, they have caught up or even pulled ahead.

Now, of course the students and families at these schools are self-selecting, and of course there may be other ways to explain away these results, and of course this is not a very large body of scholarly literature.  Nevertheless, I can't find any studies that contradict these three, and these results are consistent with thinking that what is important is not explicit instruction in discrete reading skills, and not reading a minimum proportion of informational text--but, instead, developing students minds by engaging their imaginations, creating a culture of engaged intellectual inquiry, doing lots of reading stories aloud and having them sing songs and repeat poems.

So these studies aren't definitive, but they are enough to call into further doubt the blithe assurances of people like Tim Shanahan and David Coleman that their preferred approach is consistent with the available empirical evidence.

Friday, June 14, 2013

Summer Reading Redux

I've been so busy that I haven't followed up on some stuff I posted about a long time ago and meant to come back to.  Here's a quick follow-up on one thing.

A few months ago I wrote about being on a summer reading committee, and about my cousin's seder-table diatribe against summer reading programs in which everyone reads the same book.  After that post, a few emails, and a discussion in our department meeting, I somehow ended up on a committee of two--me and a wise librarian--charged with creating a summer reading program based on (mirabile dictu) student choice.

So we made up a suggested reading list and wrote up a simple plan for students to recommend books to one other in English class this spring, read books of their own choosing over the summer (they must read at least three books) and then report back on their reading in the fall.  I'm a little worried that some of my colleagues will be pressed for time and won't bother to do the activity; I'm a little concerned that we haven't tried to provide kids with books; and I wish we had had more time to make sure everyone was on board--but at least in principle we now, at least for this year, have a summer reading program that is entirely based on students' choosing their own books.


When shown that there's no solid evidence for their side, literacy gurus demand evidence for the other

Over the past month or so, Tim Shanahan and I have been having an interesting discussion in the comments section to one of his blog posts.  I've been pleased that he's taking the time to respond thoughtfully, but he's not convincing me.  I'm writing this post, even more than some others, to clarify my own thinking--apologies for getting too much into the boring weeds here.

Initially I asked for evidence that reading more informational text led to better comprehension of such text. He said there was lots of evidence, I asked for specifics, and he finally admitted, after a few back-and-forths, that "You are correct that there is no study showing that increasing the amount of the reading of informational text has had a clear positive unambiguous impact on reading achievement or student knowledge. "

Shanahan did not, however, address why he had written in his blog post:CCSS is emphasizing the reading of literary and informational text to ensure that students are proficient with a wide variety of text. Nor did he address why, when I asked for evidence that reading more information text led to greater proficiency with informational text, he responded by saying "Actually there is quite a bit of research showing that if you want students to be able to read expository text, you have to have them read (or write) expository text."

Instead of explaining why he had made incorrect statements about the evidence for reading informational text, Shanahan asked me to show the evidence for reading literary text.  He doesn't seem to get it: my whole point is that there is not strong evidence either way, and it is dishonest to pretend that there is.  He, and many other scholars who engage in education discourse aimed at teachers and the general public, are continually pretending that there is strong scientific evidence for their pet curriculum ideas. Very often there is no such evidence.

When I suggested that a lot of "evidence-based" educational policies are not founded on particularly strong evidence, Shanahan made an interesting move: he essentially said that I was demanding too much.  As he put it, "the basic problem here is with your understanding of research and how causal claims are put forward." He said that what he and others do is to look at some available evidence and come up with a "logic model" that fits the facts.  Not all research is done, because some questions, like " Is third grade really necessary?", are not going to be studied.

So he seems to think if you have a story that is not inconsistent with some emprirically established facts, then apparently you have the right to say that "there is quite a bit of research showing" that your story is true.

Maybe.  But it seems to me that if there is debate about a question, like the question of whether it is worthwhile to make young children read more informational text, then if you say there is "quite a bit of research showing" that your side of the debate is true, you have to have evidence that is not only consistent with your side of the debate but also inconsistent with the other side.

And it's not like we couldn't do some studies!  Nell Duke, a prominent proponent of more informational text in the early grades, has gotten millions of dollars in grant money and has spent over a decade studying the issue of how much informational text children "are exposed to" in school.  Couldn't she have taken some of that large amount of time and money and done a controlled experiment?  Surely some district would have been happy to have a huge library of informational text provided to half of their K-4 schools, so that Duke could check whether students at those schools would actually do better, a few years down the line, at understanding informational texts?  But she didn't do it, and Shanahan didn't do it, and now Shanahan is implicitly suggesting that such research would be as silly as a controlled experiment in which we got rid of third grade.

I'm still trying to figure out what I think about "research-based" arguments.  I guess my position now is: research can be useful and informative, but it is only rarely, to use a legal term that has been cropping up a lot lately, dispositive; and we should have a lot more of it before we take the kind of authoritative tone that Tim Shanahan and a lot of educational experts take when they are writing for a popular audience.  In their scholarly papers, and when pressed in debate, these experts are circumspect and honest about the limitations of their certainty; I'd like to see more of that circumspection in the advice given to us teachers and to the public.