I read an interesting study yesterday. Here's my take:
The Study
A New York Times article yesterday referred to an interesting study suggesting that getting more information to parents about their kids' education is low-hanging educational fruit.
According to the study's author, Peter Bergman, a professor at Columbia, there are three basic problems in this area:
1) parents don't have high-quality information about their kids' grades and work completion
2) parental bias means that parents think their kids are doing more homework than they really are
3) the short-term bias of children (Bergman says they have "higher discount rates") leads them to conceal poor performance and poor homework completion from their parents
The literature about parental involvement has been ambiguous, according to Bergman, because it has been difficult to distinguish between cause and effect. Studies have even found negative correlations between parental monitoring/involvement and student achievement. This could either be because parental nagging is counterproductive or because parents tend to monitor and nag kids who are underperforming. Bergman thinks it's the latter.
Bergman models the situation using some fancy-looking game theory, but his experimental design is simple: the treatment group, at a California school serving mostly poor kids, got messages about missing assignments, including specific information like the date and the page number of the textbook. The control group got the default amount of information that school and teacher provided parents: report cards four times a semester and teacher contact as the teacher chose (usually very little).
Positive Results: GPA goes up; math test scores go up
The results were positive: in the treatment group, both overall GPA and math test scores increased by about .2 standard deviations. That's a pretty big increase.
This is not surprising, but it does point to an obvious lesson: schools and teachers should move to as transparent a grading system as possible. Parents should have continuous access to their children's assignments and grades. As a teacher, this will push me to be more timely and thorough about updating my gradebook, but that's a good thing. Or--it would be a good thing if I were a math teacher.
Unfortunately--as usual--English test scores don't change
Another interesting result of the study--for me, more interesting than its headline finding--was that the English test scores of the treatment group did not go up. (Well, they went up .04 standard deviations, but that's not much).
As Bergman notes, this disparity is typical--many studies find it easier to raise math scores than English scores (2). But it is nevertheless a real puzzle, one that calls into question the effectiveness of English class.
Informing parents about missing work seems to have had some direct effects. Parents who were informed about missing assignments were in better touch with the school, and more of them attended parent-teacher conferences. Parents in the treatment group took away twice as many privileges from their children (mainly access to screens and friends). Students in the treatment group skipped class less, completed more of their assignments, and attended after-school tutoring more often.
Therefore, this experiment found that a group of students who had more engaged parents, who went to class more often, who did more of their schoolwork, and who went for help after school, nevertheless did not make any more improvement on their English test scores.
This is a striking result. Going to math class and doing your math homework really matters. Going to English class and doing your assigned English work seems to matter much less.
Results like this should make English teachers very uncomfortable.
How might English teachers respond?
An English teacher at this school could interpret these results in three ways:
1) the purpose of English class is not really to improve reading ability, so this result is irrelevant, and I am going to keep doing what I'm doing.
2) the purpose of English class is to improve reading, but coming to class and doing the assigned work does not seem to serve that purpose, so I am going to change my practice.
3) the purpose of English class is to improve reading, and although coming to class and doing the assigned work does not seem to serve that purpose very well, changing my practice would be inconvenient, difficult, and unnatural, so I am going to keep doing what I'm doing.
I think most English teachers respond, if unconsciously, in the third way. Here's why:
Education is conceptualized in a way that fits math, not English
English teachers work within a system that is set up for teaching subjects, not for getting kids to read a lot. The other academic subjects, including math, are about teaching discrete concepts and techniques, while English is about continual gradual improvement in the exact same activity students have been practicing since they were in first grade. In English there is no proper "content," and the usual ways of thinking about education are inappropriate and inadequate for thinking about reading.
The usual way of thinking about education is that the teacher teaches a skill or concept, and then students practice or conceptualize on their own and get feedback from the teacher about their performance. For me, this model was most vividly represented by the Australian researcher John Hattie, who in his book Visible Learning says that the "heart of the model of successful teaching and learning" is like rappelling in that "the goals are challenging, specific and visible," the skill being taught does not come naturally, the experience of learning is "exhilarating," and "it is abundantly clear what the success criteria are". This may be a good model for learning a single skill like rappelling off the top of a building, and it may work to some extent for subjects like math, science, history or foreign languages, in which there is "material" to be learned or "skills" to be mastered, but it is not a good model for improving your performance in an activity, like reading, that you have been doing over and over for many years.
The best way for students to get better at reading is probably to spend a lot of time reading interesting material, and to spend some time discussing the meaning of interesting or difficult texts. Unfortunately, neither the reading or the discussing fits easily into education's standard conceptual framework. So instead English teachers often assign work and plan classroom activities that are neither actual reading of texts nor discussion of texts, and it doesn't really matter if our students do the work or take part in the activities.
Bergman's study shows that we should be more transparent about missing student work, but it also shows that we English teachers should be rethinking our practice.
(1)
"Parent-Child Information Frictions and Human Capital Investment:
Evidence from a Field Experiment" by Peter Bergman
(2) This phenomenon is discussed somewhat in Bergman's paper, and in more detail in a 2010 paper by Eric Bettinger. It's also touched on in a 1996 meta-analysis of summer learning loss by Cooper et al. The paper about summer learning loss is interesting, since it finds that although on average the summer learning loss is greater in math, for poor kids it is greater in reading. Food for thought.
Showing posts with label Visible Learning. Show all posts
Showing posts with label Visible Learning. Show all posts
Monday, January 19, 2015
Monday, December 10, 2012
"Visible Learning," School Reform, and In-school Reading
(I wrote this really quickly; I apologize for typos. I wanted to get it written while it was still fresh in my mind, but I have too much schoolwork to take a long time and write it well.)
In-school reading makes parents uncomfortable
At my son's middle school, and indeed throughout the district in the city my family lives in (slightly less leafstrewn than Leafstrewn, but graced with the Perfect University and the Institute of Machine Perfection), English teachers are having their students do a lot of in-class reading. On Parents' Night, I told my son's teacher I was thrilled about this. Last week, at a pot luck dinner for my daughter's class, a mom who also has a seventh grader told me she had wondered about what I said. "You said, 'I think in-class reading is a great idea,'" she said. "I couldn't tell if you were serious, or being sarcastic."
It turns out that she, and many other parents, see reading in class as a strange idea. That same week, on a parent email forum, a couple of parents raised the issue: why, they said, does school have them doing silent reading, an activity that they could easily do at home?
I wrote a response in which I made the obvious arguments: that reading is (as even such a champion of whole-class direct instruction as Doug Lemov argues) always a productive use of class time; that it is, in fact (as Lemov says), the (very high) baseline against which any other use of class time should be measured; that many kids don't read at home and indeed hardly read at all; that in-class reading allows the teacher to do more one-on-one instruction, which is the best kind; and so on.
Of the parents who responded, most backed me up. Some parents, however, were still not convinced. Most of these parents were willing to admit that reading in class might be good for other people's children--the implication I think was that good parents would have kids who read--but not for their own children, because their own children didn't need it. At the pot luck dinner I talked to a couple of parents whose kids read a lot at home; they were confused as to why their kids would spend time in school reading when they could be doing something else.
What the Research Says
I spent several hours over the weekend immersed in John Hattie's magisterial 2009 overview, Visible Learning. I've also spent some time looking back at Marzano's interesting 2000 monograph, A New Era of School Reform, which is, while less impressively encyclopedic, very much along the line that Hattie takes. Hattie and Marzano stress very clear and well-defined "learning objectives," very clear teacher control over where the lesson is going, and lots of "feedback" going both ways between teacher and student. Most education reformers over the past twenty years or so have justified their efforts by pointing to research like that which Hattie and Marzano synthesize. For a different perspective, I also looked back at Stephen Krashen's summary of studies of in-class reading, The Power of Reading. I've learned a lot, but I come away with more questions than clear answers.
Hattie and Marzano are very clear on what does work. Hattie calls it "visible learning"--he says that it's very important that both teachers and students be able to see what is working and what is not working. That requires very clear learning objectives, something Marzano stresses that seems to be taken to heart by all of today's reformers. It also requires a lot of feedback, from teachers to students, but also, and according to Hattie even more importantly, from students to teachers. Teachers need to know what their students are "getting" and what they are not getting. Teachers should be "activators" and not "facilitators." Hattie argues strongly and explicitly against the classic "constructivist" conception of teaching, in which the teacher facilitates "authentic" learning that is open-ended, "student-centered," and aimed at "discovery"--that is, learning whose end points or goals neither the teacher nor the student know beforehand. According to Hattie, this kind of "hands-off" teaching and "intrinsically motivated" learning is "almost directly opposite" to the kind of learning that the experimental data says is most successful.
Hattie and Marzano back up their arguments with vast amounts of experimental data. Hattie's book is particularly sweeping: his book is a meta-meta-analysis--that is, a meta-analysis of over 800 meta-analyses, each of which included several individual studies. Hattie's book synthesizes research on over a hundred million students, and with this vast body of evidence, a logical organization, a thoughtful, placid style but a forceful argument, it is certainly the most impressive analysis of educational data I've ever seen. That said, I came away not fully convinced.
Two Ways to Critique Hattie: theoretically and empirically
There are two kinds of potential problems with the "recipe" that Hattie (along with Marzano and all the other proponents of "clear learning objectives") is selling.
One kind of problem is theoretical: it seems possible that this kind of learning is potentially dry, uninspiring, and limited--and perhaps especially boring to students who get the idea quickly; also, it's not clear to me that, in the case of reading, the "learning goals" can be, to use Hattie's words, "challenging, specific and visible" (25).
A second class of problems, that I'll deal with in a post later this week, have to do with the empirical evidence Hattie is using: (a) the evidence for this recipe's efficacy seems to be quite thin in the case of reading; (b) the evidence for this recipe's efficacy seems to be mostly short-term, not long-term; (c) Hattie's evidence is mostly from controlled experiments, not from natural experiments; (d) Hattie and especially Marzano argue loudly that their evidence shows that poverty can be overcome by good schools and good teaching, but as far as I know this has never actually been done by an actual school in the real world--or at least not in the United States--despite the fact that thousands of schools across the country are trying to put this kind of "visible learning" into practice.
A theoretical critique: Is "Visible Learning" possible in English class--and if so, is it doomed to be dry and boring?
Today I wrote the following on the board in my "Honors" American Lit. class: "Learning Targets: (1) I will be able to write a good question about a sophisticated text and find a specific passage through which I can explore the question; (2) I will be able to discuss coherently the relationship between Hawthorne and Transcendentalism."
The cluster of A students who sit near the front of the room noticed what I had written and hooted derisively. One of them asked, "Is that a joke?" Another said, "No teacher in my entire school career has ever written learning targets on the board." Another one said it reminded him of a description of a typical day at a Fall River charter school that was posted a few days ago on Edushyster, an anti-corporate Ed reform blog. This led to a spirited discussion for a minute or so, before I shut it down so that we could get to our learning targets.
One student said that she liked the idea of learning targets, even in English class, but the most common knee-jerk response from the kids was that putting up a "learning target" every day in English class was laughable and ridiculous. Again, we didn't spend much time on this, but I believe their reaction was based on two ideas that I've written about before: (1) that, on the micro level, the skills we work on in English class are very complex and interdependent, so that isolating one of them each day is either absurd (today we're learning about commas in Hemingway) or obvious (the one I put up), while on the macro level the skills are (2) daily learning goals promote short-term thinking, while English-class skills are gained over the very long term--learned through repeated practice over years.
What are the learning goals in the case of upper-grade reading?
It is worth asking, then, what people who support "visible learning" would say should be our learning goals. The obvious place to look for these learning goals is in the Common Core.
The Common Core has two reading strands for grades 6-12, one covering "Literature" and one covering "Informational Text." I looked at the Literature strand. The Common Core specifies 9 standards for literature in the ninth and tenth grade. Three of them are below:
CCSS.ELA-Literacy.RL.9-10.2 Determine a theme or central idea of a text and analyze in detail its development over the course of the text, including how it emerges and is shaped and refined by specific details; provide an objective summary of the text.
CCSS.ELA-Literacy.RL.9-10.4 Determine the meaning of words and phrases as they are used in the text, including figurative and connotative meanings; analyze the cumulative impact of specific word choices on meaning and tone (e.g., how the language evokes a sense of time and place; how it sets a formal or informal tone).
These standards seem fairly unhelpful. They are essentially the same as standards from earlier grades (1). That would be fine in itself; as I've written, English class is a matter of practicing the same skills over and over. The problem is that these standards don't tell us anything we don't already know. We know that every English student in the world will be required to "determine the meaning of words and phrases as they are used in the text." This is quite a low-level goal.
In the end, I don't think putting a "learning target" up on the board is a particularly bad idea. I don't think it's necessary, but in the hands of a decent teacher it won't necessarily hurt. The danger with the practice is that it may lead to reductive, simplistic, boring teaching, in which kids are taught a lesson completely focusing on the simple structure of Hemingway's sentences. On the other hand, the danger with the opposite approach, the let's-just-read--and-talk-about-it approach, is that it may lead to a class that is way too loose. Either way is dangerous, but it seems unlikely that telling kids specifically what they are going to learn in every lesson is going to lead to more learning overall. Learning in English class is, as the Common Core standards admit both implicitly and (sometimes) explicitly, cumulative and long-term. To pretend that students are going to be noticeably better at "determining the meaning of words and phrases as they are used in the text" after a day-long lesson, or even after a month-long unit, is silly. Determining the meaning of words and phrases is a life-long task--and it is one at which the learning curve becomes flatter and flatter. I am probably better in some ways at this now than I was when I was twenty-five; but I'm not sure anyone would notice the improvement. Does this mean that reading a new author (as when I discovered Edward St. Aubyn last year) is useless? No, of course not. But it would be silly to set targets for that learning ("I will be able to describe Patrick Melrose's family dysfunction...").
That may mean that more advanced students will be the more dismissive of "visible learning"; perhaps, but I think it's not necessarily great for any students, and I'll talk more about that in a post on the empirical critique of Hattie's work later this week.
********************************************************************************
Footnote:
(1)
The fourth grade standards include the following:
CCSS.ELA-Literacy.RL.4.2 Determine a theme of a story, drama, or poem from details in the text; summarize the text.
CCSS.ELA-Literacy.RL.4.4 Determine the meaning of words and phrases as they are used in a text, including those that allude to significant characters found in mythology (e.g., Herculean).
In-school reading makes parents uncomfortable
At my son's middle school, and indeed throughout the district in the city my family lives in (slightly less leafstrewn than Leafstrewn, but graced with the Perfect University and the Institute of Machine Perfection), English teachers are having their students do a lot of in-class reading. On Parents' Night, I told my son's teacher I was thrilled about this. Last week, at a pot luck dinner for my daughter's class, a mom who also has a seventh grader told me she had wondered about what I said. "You said, 'I think in-class reading is a great idea,'" she said. "I couldn't tell if you were serious, or being sarcastic."
It turns out that she, and many other parents, see reading in class as a strange idea. That same week, on a parent email forum, a couple of parents raised the issue: why, they said, does school have them doing silent reading, an activity that they could easily do at home?
I wrote a response in which I made the obvious arguments: that reading is (as even such a champion of whole-class direct instruction as Doug Lemov argues) always a productive use of class time; that it is, in fact (as Lemov says), the (very high) baseline against which any other use of class time should be measured; that many kids don't read at home and indeed hardly read at all; that in-class reading allows the teacher to do more one-on-one instruction, which is the best kind; and so on.
Of the parents who responded, most backed me up. Some parents, however, were still not convinced. Most of these parents were willing to admit that reading in class might be good for other people's children--the implication I think was that good parents would have kids who read--but not for their own children, because their own children didn't need it. At the pot luck dinner I talked to a couple of parents whose kids read a lot at home; they were confused as to why their kids would spend time in school reading when they could be doing something else.
What the Research Says
I spent several hours over the weekend immersed in John Hattie's magisterial 2009 overview, Visible Learning. I've also spent some time looking back at Marzano's interesting 2000 monograph, A New Era of School Reform, which is, while less impressively encyclopedic, very much along the line that Hattie takes. Hattie and Marzano stress very clear and well-defined "learning objectives," very clear teacher control over where the lesson is going, and lots of "feedback" going both ways between teacher and student. Most education reformers over the past twenty years or so have justified their efforts by pointing to research like that which Hattie and Marzano synthesize. For a different perspective, I also looked back at Stephen Krashen's summary of studies of in-class reading, The Power of Reading. I've learned a lot, but I come away with more questions than clear answers.
Hattie and Marzano are very clear on what does work. Hattie calls it "visible learning"--he says that it's very important that both teachers and students be able to see what is working and what is not working. That requires very clear learning objectives, something Marzano stresses that seems to be taken to heart by all of today's reformers. It also requires a lot of feedback, from teachers to students, but also, and according to Hattie even more importantly, from students to teachers. Teachers need to know what their students are "getting" and what they are not getting. Teachers should be "activators" and not "facilitators." Hattie argues strongly and explicitly against the classic "constructivist" conception of teaching, in which the teacher facilitates "authentic" learning that is open-ended, "student-centered," and aimed at "discovery"--that is, learning whose end points or goals neither the teacher nor the student know beforehand. According to Hattie, this kind of "hands-off" teaching and "intrinsically motivated" learning is "almost directly opposite" to the kind of learning that the experimental data says is most successful.
Hattie and Marzano back up their arguments with vast amounts of experimental data. Hattie's book is particularly sweeping: his book is a meta-meta-analysis--that is, a meta-analysis of over 800 meta-analyses, each of which included several individual studies. Hattie's book synthesizes research on over a hundred million students, and with this vast body of evidence, a logical organization, a thoughtful, placid style but a forceful argument, it is certainly the most impressive analysis of educational data I've ever seen. That said, I came away not fully convinced.
Two Ways to Critique Hattie: theoretically and empirically
There are two kinds of potential problems with the "recipe" that Hattie (along with Marzano and all the other proponents of "clear learning objectives") is selling.
One kind of problem is theoretical: it seems possible that this kind of learning is potentially dry, uninspiring, and limited--and perhaps especially boring to students who get the idea quickly; also, it's not clear to me that, in the case of reading, the "learning goals" can be, to use Hattie's words, "challenging, specific and visible" (25).
A second class of problems, that I'll deal with in a post later this week, have to do with the empirical evidence Hattie is using: (a) the evidence for this recipe's efficacy seems to be quite thin in the case of reading; (b) the evidence for this recipe's efficacy seems to be mostly short-term, not long-term; (c) Hattie's evidence is mostly from controlled experiments, not from natural experiments; (d) Hattie and especially Marzano argue loudly that their evidence shows that poverty can be overcome by good schools and good teaching, but as far as I know this has never actually been done by an actual school in the real world--or at least not in the United States--despite the fact that thousands of schools across the country are trying to put this kind of "visible learning" into practice.
A theoretical critique: Is "Visible Learning" possible in English class--and if so, is it doomed to be dry and boring?
Today I wrote the following on the board in my "Honors" American Lit. class: "Learning Targets: (1) I will be able to write a good question about a sophisticated text and find a specific passage through which I can explore the question; (2) I will be able to discuss coherently the relationship between Hawthorne and Transcendentalism."
The cluster of A students who sit near the front of the room noticed what I had written and hooted derisively. One of them asked, "Is that a joke?" Another said, "No teacher in my entire school career has ever written learning targets on the board." Another one said it reminded him of a description of a typical day at a Fall River charter school that was posted a few days ago on Edushyster, an anti-corporate Ed reform blog. This led to a spirited discussion for a minute or so, before I shut it down so that we could get to our learning targets.
One student said that she liked the idea of learning targets, even in English class, but the most common knee-jerk response from the kids was that putting up a "learning target" every day in English class was laughable and ridiculous. Again, we didn't spend much time on this, but I believe their reaction was based on two ideas that I've written about before: (1) that, on the micro level, the skills we work on in English class are very complex and interdependent, so that isolating one of them each day is either absurd (today we're learning about commas in Hemingway) or obvious (the one I put up), while on the macro level the skills are (2) daily learning goals promote short-term thinking, while English-class skills are gained over the very long term--learned through repeated practice over years.
What are the learning goals in the case of upper-grade reading?
It is worth asking, then, what people who support "visible learning" would say should be our learning goals. The obvious place to look for these learning goals is in the Common Core.
The Common Core has two reading strands for grades 6-12, one covering "Literature" and one covering "Informational Text." I looked at the Literature strand. The Common Core specifies 9 standards for literature in the ninth and tenth grade. Three of them are below:
CCSS.ELA-Literacy.RL.9-10.2 Determine a theme or central idea of a text and analyze in detail its development over the course of the text, including how it emerges and is shaped and refined by specific details; provide an objective summary of the text.
CCSS.ELA-Literacy.RL.9-10.4 Determine the meaning of words and phrases as they are used in the text, including figurative and connotative meanings; analyze the cumulative impact of specific word choices on meaning and tone (e.g., how the language evokes a sense of time and place; how it sets a formal or informal tone).
These standards seem fairly unhelpful. They are essentially the same as standards from earlier grades (1). That would be fine in itself; as I've written, English class is a matter of practicing the same skills over and over. The problem is that these standards don't tell us anything we don't already know. We know that every English student in the world will be required to "determine the meaning of words and phrases as they are used in the text." This is quite a low-level goal.
In the end, I don't think putting a "learning target" up on the board is a particularly bad idea. I don't think it's necessary, but in the hands of a decent teacher it won't necessarily hurt. The danger with the practice is that it may lead to reductive, simplistic, boring teaching, in which kids are taught a lesson completely focusing on the simple structure of Hemingway's sentences. On the other hand, the danger with the opposite approach, the let's-just-read--and-talk-about-it approach, is that it may lead to a class that is way too loose. Either way is dangerous, but it seems unlikely that telling kids specifically what they are going to learn in every lesson is going to lead to more learning overall. Learning in English class is, as the Common Core standards admit both implicitly and (sometimes) explicitly, cumulative and long-term. To pretend that students are going to be noticeably better at "determining the meaning of words and phrases as they are used in the text" after a day-long lesson, or even after a month-long unit, is silly. Determining the meaning of words and phrases is a life-long task--and it is one at which the learning curve becomes flatter and flatter. I am probably better in some ways at this now than I was when I was twenty-five; but I'm not sure anyone would notice the improvement. Does this mean that reading a new author (as when I discovered Edward St. Aubyn last year) is useless? No, of course not. But it would be silly to set targets for that learning ("I will be able to describe Patrick Melrose's family dysfunction...").
That may mean that more advanced students will be the more dismissive of "visible learning"; perhaps, but I think it's not necessarily great for any students, and I'll talk more about that in a post on the empirical critique of Hattie's work later this week.
********************************************************************************
Footnote:
(1)
The fourth grade standards include the following:
CCSS.ELA-Literacy.RL.4.2 Determine a theme of a story, drama, or poem from details in the text; summarize the text.
CCSS.ELA-Literacy.RL.4.4 Determine the meaning of words and phrases as they are used in a text, including those that allude to significant characters found in mythology (e.g., Herculean).
Subscribe to:
Posts (Atom)