Recording reading conferences

A month ago I wrote about reading conferences, and I said that I wasn't sure how to keep good records of the discussions I had with students.  Some of my brilliant colleagues, who have been thinking about these kinds of discussions too, have started recording their conferences using a voice recorder. Today I started doing this too, using my ipod.  I have the kids read to me for 30 seconds or a minute from wherever they are in the book they're reading independently, and then I ask them some questions.  I usually start with something like, "What do you think the author is doing in this passage?"  I try to follow up with "What specifics in the passage do you see that make you say that?"  And then we go from there.

A few thoughts after my first day:

1) It's a good thing we get used early to the look of ourselves in mirrors; I'm not yet used to the sound of my voice on a recording.

2) Maybe if I did this more my voice would improve?

3) All of my students have a fair amount to say about what they're reading--that's good!

4) I wonder if they would have as much to say if they were talking about a whole-class text.  I suspect that talking about the independent reading book makes the students the experts, and therefore empowers them--but I don't have a control group, so I don't really know.

5) It takes longer than I realized to get at an important question, or to get to something that makes the students stop and think.  When I wrote about this a month ago, it seemed easy and quick to get to those points; I think I had had an exceptionally good day of conferences just before I wrote that post. (I wrote, "One thing that's striking in doing these one-on-one conversations is how quickly we get to points at which the students need to stop and think before they respond.)  Listening to today's recordings, I am struck by how relatively smooth the conversations are.

6) I wonder if this is partly because in today's conversations I allowed the student to direct the conversations more than I often do.  A couple of days ago I was talking to colleagues about conferences, and we discussed using open-ended questions, so today I tended to start the same way with everyone, and I didn't have the explicit goal of coming up with a question that made the student stop and think. This way of questioning, which was partly modeled on the VTS method ("Visual Thinking Strategies"), showed me more clearly what the student was capable of on her own, but didn't lead to the "stop and think" moments that last month I was apparently so proud of facilitating.

6) Nevertheless, I do get something out of these conversations--they are possibly useful assessment tools.  From conversations today I learned that: student A doesn't know the historical background, so can't get the humor and nuance of the conversation; student B almost never refers back to the text, even when asked repeatedly about "specifics in the passage"; student C makes great connections between this scene and other parts of the book; students B, D and F are not very fluent or accurate in their reading aloud, but B and D nevertheless seem to understand the passage perfectly; student E doesn't seem to remember anything from the book except what he's just read to me; etc.

7) It seems to me that these conversations might be pretty good for diagnosing issues--and for instruction, as I was thinking before--but again, as with all instruction, it is SO INEFFICIENT!

8) My main instructional function in these conferences seems to be to push them toward a closer attention to the text and toward deeper thinking. This is really difficult!

9) I still haven't figured out how to keep a good record of these conferences.  In one sense I have excellent records--I have audio recordings--but in another sense these records are uselessly unwieldy.  From 13 conferences I have about a little over an hour of audio.  I did not try to take notes at the same time--but I probably should, so that I can later check my notes against the audio and improve my note taking...

10) I am still haunted by the sheer slipperiness of trying to improve reading comprehension.  I do think that making kids look more closely at text is a worthwhile exercise, but I also wonder how much my own reading comprehension "skills" have improved in the years since I was 12 or 13.  I know much more (background knowledge, vocabulary, literary terms, etc.), but are my actual reading skills (questioning, inferring, making connections) any better?  I am reading an apparently excellent 2005 overview of the research on comprehension acquisition (Perfetti, Landi, Oakhill), and it seems that their recommendations for instruction center on: 1) Reading more (and making sure the reading is "successful"--i.e. comprehensible input); (2) Instruction that tries in various ways to get students to pay more attention to the text and to their own understanding of it ("monitoring").  I take some comfort in thinking that reading conferences are one way to try to do #2--and that while I'm conferencing, the rest of the kids are doing #1.

I look forward to recording more conversations with students about what they're reading next week, and trying to put together the data into some kind of coherent record.  For now, I'm going to just try to get used to the sound of my own voice.

Conversations and discussions as (formative) Reading Assessments

After spending a few days thinking about little besides snow, I want to take an hour and try to write about reading assessment, and in particular about the kind of natural assessment that can happen in a conversation about a book.

All assessment is formative, and all assessment is also instruction
The only valid educational reason for assessment is to improve education; therefore, anything we think of as "summative" should probably be re-engineered to make it more formative. In addition, assessment is always, inevitably, teaching something, and we should consider, before we assess, what our assessment will teach.  But that's abstract...

Conversation and discussion as assessment
One good way to assess student reading is through conversation and discussion.  In a discussion about a text she has read, the student should get a lot of truly immediate feedback about her reading. 

While my students are reading in class, I go around and talk to individual students.  Sometimes these conversations are on a specific topic common to the whole class: in our grammar unit, for instance, I will have quick check-ins with kids about, say, prepositional phrases on the page they are reading; if I've assigned the class an essay about point of view, we will talk about that.  At other times, however, the conversations are totally individualized, organic exchanges in the style (I think) of Nancie Atwell, about particular issues in the books they are reading on their own, and sometimes about particular passages from these books (I usually stick to the passage they happen to be on just then).

Usually I have not read these books--and if I have read them, I don't remember the details all that well--so the student is of necessity playing the role of the expert.  This is a good role for a student to play in an assessment, since it empowers her to put on her best performance.  If, as in most classroom situations, not to mention most assessments, the student is not the expert, then she is put in a position of wanting to say what the teacher thinks is right, and it's hard for either the student or her teacher to get a sense of the quality of her own independent thought.

When I have a conversation like this, I think my job is to ask the student questions that will get at the quality of her reading and make her thinking more visible to herself and to me--so that she can see what she knows, how she knows it, and what she doesn't know.  I ask different questions depending on the student.  Sometimes I start out with a general question.  If the student is just starting a book, I might ask, "Are you liking it so far?" Whether the answer is yes or no, I might ask, "Why?"  Or: "What's good about it?" Sometimes, instead of starting with a question, I start by having the student read to me for a half a page or so.  This is another kind of assessment, and one I don't even have to give feedback on--the kid can hear, herself, where she is stumbling.

When students have to stop and think
 As I follow up, either on my original question or on the passage the kid read to me, the questions narrow to a particular focus, and very quickly we get to a question that the student doesn't know the answer to already.  In some cases these questions are ones that I would expect to be obvious, like, "Why is the main character so angry at her friend?" In that case, the student's confusion is very interesting, since we would seem to be identifying a basic problem with comprehension, yet doing so in a way that might seem inherently interesting, and in a way that encourages the student to find out the answer, rather than, as on a standardized assessment, putting the emphasis on the student's failure to figure out what the teacher or state already know, and with little opportunity for immediate follow-through.

In other cases the questions that give the student pause are more difficult or literary, like "Why do you think the author started the book with this scene?" or "How could you tell that she was angry?"  In these cases, too, we are noticing what the student has already considered and what she has not given a thought to.  (At other times, the students get something obviously wrong, and the teacher can follow up in a gentle and friendly way and allow the student to figure out for herself what she was confused about and why.)

One thing that's striking in doing these one-on-one conversations is how quickly we get to points at which the students need to stop and think before they respond.  This stopping and thinking is a pretty big difference between thoughtful intellectual conversations and the usual adolescent repartee. As I remember from my own youth and observe in my students, adolescent conversations are mainly about loud, immediate disagreement or loud, immediate agreement. The loudness and immediacy overwhelms most of the potential for thoughtful critical analysis.  One-on-one conversations in the classroom, conducted in a whisper and aiming less at feel-good agreement or dramatic disagreement, are dramatically different, not least because they lead to so much stopping and thinking.

My guess is that this stopping and thinking is when much of the learning happens, as students see what they understand and what they don't, and as they think through new ideas that they haven't thought about before.  For the conversation is not only assessment, but is also a form of instruction.  In my questioning I am instructing them in ways of looking at a book, in categories of literary thought, in literary vocabulary, and so on.

Disadvantages of this method of assessment
The major disadvantage of these individual conversations is that each student can't get very much of my time.  If it takes a couple of minutes for the class as a whole to settle down enough for me to start talking to kids individually, and if each conversation takes four minutes, and if also I want to quickly check on what progress the other kids in the class have made, then I can get through four conversations in a twenty-minute independent reading period.  That means, for my sixteen-student classes, that I can talk to each kid individually for four minutes each week.  That's not very efficient.

Another disadvantage is that the assessment is not uniform.  I'm not checking each kid against the same benchmark, so it's not easy to compare.  Another disadvantage is that these assessments are often random, coming organically out of whatever passage the kid happens to be reading right then.  Of course, these two disadvantages are also advantages, since the lack of uniformity means that the assessments are better suited to the individual students and the randomness of the passages often sparks my thinking in ways that I couldn't have anticipated.

A last disadvantage is that it's been hard, at least for me, to keep good records of this kind of qualitative, individualized assessment, so it's hard to measure progress and to follow up.  I have to confess that in my preliminary experiments with this kind of assessment, I haven't yet figured out a good record keeping system.  It needs to be very simple, because I, like Ben Franklin, am not very organized.  I'm going to work on this over the next month or so, and I'll follow up with another post, in which I also give some more specific examples of these kinds of conversations.

Conclusion
Most people think about assessment in the same way Mr. Google does (google "reading assessment" to see what I mean): that is, as standardized tests, usually written, administered, by all-knowing adult authorities, upon children who are probably all too aware that (1) they're being tested and (2) that the assessment is of very little interest to either the adult or kid except as an assessment. So perhaps the best thing about using an informal conversation about an independent reading book as an assessment is that it doesn't feel like an assessment.  All of what I'm saying here seems incredibly obvious--probably even when Rousseau said it it seemed pretty obvious--but it might be worth reminding ourselves that assessment is about more than just testing.

*****************************************************************
Post Script: Similarity to what happens naturally in a literate family; limitations of school
The kind of conversations I've discussed are essentially like the conversations that we have with children in our own homes, starting with the conversations we had when we were reading picture books to them.  The fact that I think doing this in a classroom for four minutes a week is worthwhile is quite amazing, given that many four-year-olds get this kind of treatment for twenty minutes every single night.

This points, perhaps, to the limitations of school.  There's no way that I can possibly do as good a job, as an English teacher responsible for the reading and writing of 85 children, as I do as a father responsible for the reading and writing of two children.  In a sense that's okay--as long as what they do with me is worthwhile--but it's worth remembering the limitations of the system in which we work.

My colleagues and I gave a common assessment!

Leafstrewn is renowned for its independent and distinctive teachers.  Or, some might say, we're notorious for our lack of consistency.  In any case, every teacher has her own way of doing things.  We can’t agree on common policies and practices, let alone enforce them. If ten people discuss an issue, there will be passionate defenses of twenty different opinions.  Our English department chair uses the phrase “herding cats” more frequently than she probably realizes.  So when the ninth grade teachers—all ten of us—decided to make half of our midyear exam a common assessment, I wasn’t sure if it would actually happen.

Amazingly enough, we did it.  Every freshman at Leafstrewn was given the same two-page passage from a Sherman Alexie story and the same prompt for an analytical paragraph, and every paragraph was graded with the same 5-category rubric.  To make the grading more objective, we graded the exams, not of our own students, but of our colleagues’.

We will eventually sit down and consider the numbers, but I have a few reactions now:

(1) reading a passage and writing an analysis of it is an extraordinarily complex task, which is great, but it’s pretty hard to assess in an objective way;

(2) the strength of rubrics is that they are specific and explicit, but this is also their weakness;

(3) everyone got a B;

(4) that’s okay!

(5) I could have done a much better job of preparing my students, and that preparation would have been better not just for this assessment, but in general.

(1) Reading a passage and writing an analysis of it is an extraordinarily complex task, which is great, but it’s pretty hard to assess in an objective way

There is an incredible amount to keep track of when you’re reading anything closely—emotions, connotations of particular words, figures of speech, intertextuality, patterns (like repetition and contrast) within the text, etc.  Writing, too, is really, really complicated—you have to master grammar, ideas, structure, logical arguments, relevant evidence, and so on.  Skilled readers and writers do all this unconsciously, and we sometimes forget that it is amazingly complicated, and our brains, even the most “limited” of them, are quite incredible.

This incredible  complexity becomes much more visible when you start talking about how to judge the quality of student reading and writing.  Different teachers have different ideas of which pieces of this incredible complexity to focus on.  It’s like the story of the elephant: I’m looking at the elephant’s legs, one colleague is looking at the trunk, another is bumping into the tusks, and so on.  Designing a rubric is tough, because there are always things that you’re leaving out, or looking at from only one side.

(2) The strength of rubrics is that they are specific and explicit, but this is also their weakness

Much of the trouble in grading the assessment came in using the rubric.  A rubric is intended to make the grading more transparent and clear, and, most crucially, more specific.  If a student is told, “Your essay is bad,” the student will want to know what in particular was bad about it.  A rubric is supposed to offer the kid that specificity.  What was interesting about the grading process was that the specificity of the rubric was usually exactly what caused trouble in the grading.

For example, if we judged that a conclusion is not good enough for the “Good” category, we had to circle the box on the rubric for a conclusion that “Needs Work.” That box reads: “Brings paragraph to a finish that repeats previous ideas.”  The problem is that there are many ways for a conclusion to be bad, and repeating previous ideas is only one of them. I ran into this problem of overly specific descriptors in every single category on this rubric.

(3) Everyone got a B

Either because our rubric was too easy at the low end, too tough at the high end, or because our students are all pretty good, or because we did a good job of preparing our students, most of the grades fell in the B range. 

(4) That’s okay!

I think one of the lessons here is that actually for all our hand-wringing, our kids are really quite competent.  They can read a passage and write a reasonable paragraph about it.  They are not illiterate.  Almost all of them managed to come up with identifiable topic sentences, evidence that more or less supported their main ideas, and a conclusion that in some way related to what they were saying.  This is no mean feat for a fourteen year old, and I wonder, I admit, if it has something to do with MCAS.  Maybe, as our department chair says, MCAS has really improved kids’ ability to write these kinds of paragraphs.

(5) I could have done a much better job of preparing my students, and that preparation would have been better not just for this assessment, but in general!

I think this common assessment was a great thing to do.  Having students read something and analyze it in a disciplined way is worthwhile, and doing it as a group certainly made my own teaching better.  I was more focused, my students were more motivated, and it took some of the dissonance out of the grading.  (Normally, when we grade our own students’ work, there is an uncomfortable dissonance.  It is as if Bela Karolyi were to judge his own gymnasts’ routines, or as if a soccer coach were training his team for a game against herself.)  In a fairly short and stress-free preparation, I think I did a reasonably good job.

Nevertheless, although my students' performance was fine, there was a lot of room for improvement. How could I have helped them more?  What could I have done better?  A bunch of things, but here's one: I didn’t train my students well enough in coming up with a good main idea.  They tended to say something like, “The impression the author creates in this impression is of a family that is struggling.”  That is pretty obvious, and I need to help my students learn how to go deeper. To take an obvious thought and push it deeper one may:

  * Explore the why of the obvious thought (e.g. the family struggles because they’re in denial).

  * Consider ways in which the opposite is true as well, and craft a semi-dialectical topic sentence of the "Although A, nevertheless B" type--and then by the end of their paper they may arrive at the synthesis of C.  Later in life they can worry, Mr. Ramsay-like, about getting to Q or Z.

  * Explore the how of the obvious thought (e.g. the family struggles ineffectually, trying the same things over and over again even though they produce no results (father looking in wallet over and over, son dreaming over and over, etc.))

* Are there other ways?  Applying a schema? Making a connection?  What else? 

Teaching my students to push their thinking further would be useful not merely for the exam, but in general.  This is something that would be useful to focus on explicitly, and that I somehow overlooked.  That is one of the virtues of this common assessment--it makes the whole process more conscious and transparent, and so lets us see things that we should have seen before.

In the end, though, we and our students did a fine job.  Now if we could just get them to like to read...