I’m a fan of PBS’s John Merrow.  He is the rare television journalist—perhaps the only one—who has the interest, background and sufficient airtime to give thorny education topics the nuanced treatment they deserve.  The other night he devoted nearly ten minutes of PBS’s Newshour to an intriguing question:  Can a good school have bad test scores?  To answer it, he and his producer Cat McGrath visited P.S. 1, a South Bronx school that appears to be filled with bright, eager learners and devoted teachers, yet is “failing” as judged by its terrible reading scores.  What’s going on?

“We discovered that the FIRST graders at that school were reading confidently and competently, Merrow writes on his blog, “but the fourth graders weren’t according to the results of the state test. Is this a paradox, or a full-blown contradiction?”  Merrow attempted to figure out where things leave the rails between first and fourth gradesan earnest, but ultimately frustrating piece that correctly diagnoses the problem, but fails to uncover or sufficiently examine its root causes.

Merrow starts by correctly pointing out that there is a big difference between “reading” in the first grade and “reading” in fourth grade.  Indeed, they’re hardly the same activity. Observing a phonics lesson in a first grade classroom, he points out that “Ms. Hunts students seem to be getting it. What they are doing is called decoding, but decoding is only half the battle. Understanding what the words mean is a much harder skill called comprehension. Its where many children fall flat.”

For starters comprehension is not a “skill” at all.  Your ability to read with comprehension depends on many things.  You must be able to decode.  You must know all of the words.  And you must know at least a little about the subject matter of the text to construct a mental model that allows you to make meaning correctly.   “My dog is sleeping on the couch” is easy to understand; “ My Havanese is snoozing on the divan” means more or less the same thing as long as you know about dogs and furniture, and understand that snooze is a synonym for sleep. 

Only 18% of P.S 1s 4th graders are reading at or above grade level.  The good decoders have failed to become strong readers. What happened?  One 4^th grade teacher says the children’s home lives start to take a toll.

“Theyre not as innocent anymore. Theyre realizing the things that are affecting their schoolwork. You know, I mean, I have homeless students in my room. I have students with fathers in jail. Theres drugs. So, that obviously comes into play at a certain point as well.”

Another 4^th grade teacher suggests the grind of test prep and test anxiety is the issue.  “The system takes the fun out of reading,” observes Brenda Cartagena.  “I want them to read for enjoyment. I want them to grab that book because its fun. I tell them, reading, you travel, you meet new friends, you learn how to do new things. But its very difficult, you know? They take the joy out. And its hard to infuse it back.”

Full disclosure: I spent a significant amount of time talking to producer McGrath as she and Merrow prepared the piece.  I stressed the importance of vocabulary and background knowledge and how reading comprehension, unlike decoding, is not a transferable “skill” at all.  How the tests children take are de facto tests of general knowledge.  To what degree, I wondered, does the instruction these South Bronx kids receive reflect that?  Having taught at a nearby school in the same district a few blocks away, I suspected the answer is “not at all.”   To their credit Merrow and McGrath looked at the tests.  Merrow writes on his blog:

“We looked over past tests, and, sure enough, the passages were about subjects that poor kids in the south Bronx may not be familiar with . Answering the questions did require inferential leaps, just as we had been told.

“So we asked to talk with a couple of fourth graders who were reading below grade level, and here’s where it got complicated.  As you will see in the NewsHour piece, both children, one age 9 and the other 11, handled the passages and answered all the questions. Maybe the personal attention helped, but they read easily and drew inferences correctly. We only ‘tested’ a couple of kids, but both were below grade-level, their teacher assured us.”

Again, did they “read easily?”  Or did they decode easily?  And Im not as confident as Merrow that they drew inferences correctly.  Heres what viewers saw Monday night on the Newshour:

JOHN MERROW: I wondered how the fourth-grade class might perform on the state test this year, and asked Ms. Cartagena to send me two of her students who were reading below great level.

Jeannette, who is 9, came first.

STUDENT: So far, I have hoped to find many new species.

JOHN MERROW: I asked her to read a passage about dragonflies from last years state test.

STUDENT: About 5,500 dragonfly species buzz around the world. Who doesnt like like looking at these amazing insects?

JOHN MERROW: What are species?

BRENDA CARTAGENA: Many kinds.

JOHN MERROW: Kinds. Its kinds of species. Right. Exactly. Yes.

Exactly right?  It is impossible to know, based on this exchange, if the child understands “species” as well as Merrow assumes or if she has a sufficient grasp of what a dragonfly is to apply the concept.  As a teacher, I’d want to probe more for understanding, “if you’re looking at two dragonflies, how can you tell if they are different species?” you might ask.  If she said they might be different colors or have different shaped wings, I’d feel reasonably confident that she understands the basic idea.  If she says “one’s male and one’s female” or can’t explain the difference at all, then the concept is still shaky, or she might not know enough about dragonflies to apply it. Either way it would impact her ability to draw inferences and make meaning from the passage.

Given that the achievement gap long predates test-driven accountability, you could sensibly argue that that testing makes the problem worse, but it cannot be the root cause.  Similarly, the idea that “real life catches up with kids” by 4^th grade is unsatisfying.  If reading comprehension is a skill like riding a bike or throwing a ball through a hoop , it is not an ability you would suddenly lose if your father was sent to prison or you were evicted from your home.

What Merrow either didn’t probe or did not air is what – what exactly – the instruction given to these children in 2nd, 3rd and 4th grade looks like.  Are they being steeped in a content-rich curriculum that would make it less likely that concepts like cicadas, dragonflies and species would be unfamiliar at test time?  Or is the school operating, as most do, on the incorrect  assumption that reading comprehension is a transferable skill?  That decoding + engagement + is enough to guarantee success? When this formula fails, as it inevitably must, it is normal to look to outside causes like poverty, fractured families and test anxiety as root causes.  These things certainly work against student engagement and achievement, but they are clearly not the cause of failure. 

Merrow is due a lot of credit for  taking a nuanced view of reading and and asking the right question: why doesn’t early decoding success automatically turn in to comprehension success?  But ultimately the piece doesnt provide the answer.

Similar Posts:

• Little change on statewide writing test scores
• Hacking at Branches
• Unforgettable Learning
• What We Need is a “Prep” Rally
• Teach Now, Test Later