“A noted Williams College psychologist argues standardized tests are useful, if they measure the abilities students really need.
By Susan Engel FEBRUARY 26, 2015
In the past few years, parents, teachers, and policy makers have furiously debated whether standardized tests should be used to promote or hold back children, fire teachers, and withhold funds from schools. The debate has focused for the most part on whether the tests are being used in unfair ways. But almost no one has publicly questioned a fundamental assumption — that the tests measure something meaningful or predict something significant beyond themselves.
I have reviewed more than 300 studies of K–12 academic tests. What I have discovered is startling. Most tests used to evaluate students, teachers, and school districts predict almost nothing except the likelihood of achieving similar scores on subsequent tests. I have found virtually no research demonstrating a relationship between those tests and measures of thinking or life outcomes.
When you hear people debate the use of tests in schools, the talk usually assumes that the only alternative to the current approach is no testing at all. But nothing could be further from the truth. Ideally, everyone would benefit from objective measures of children’s learning in schools. The answer is not to abandon testing, but to measure the things we most value, and find good ways to do that. How silly to measure a child’s ability to parse a sentence or solve certain kinds of math problems if in fact those measures don’t predict anything important about the child or lead to better teaching practices.
Why not test the things we value, and test them in a way that provides us with an accurate picture of what children really do, not what they can do under the most constrained circumstances after the most constrained test preparation? Nor should this be very difficult. After all, in the past 50 years economists and psychologists have found ways to measure things as subtle and dynamic as the mechanisms that explain when and why we give in to impulse, the forces that govern our moral choices, and the thought processes that underlie unconscious stereotyping.
Here are seven abilities and dispositions that kids should acquire or improve upon — and therefore should be measured — while in school. One key feature of the system I am suggesting is that it depends, like good research, on representative samples rather than on testing every child every year. We’d use less data, to better effect, and free up the hours, days, and weeks now spent on standardized test prep and the tests themselves, time that could be spent on real teaching and learning.
Every child should be able to read by the end of elementary school. Just as important, every child should be reading on a regular basis, turning to books and other written material for pleasure and for information. What does it mean to be able to read? It means having the ability to read an essay or book and understand it well enough to use the information in some practical way or to talk about it with another person.
When children can and do read, their language and thinking are different. One way to measure reading, then, is to take a close look at their language and thinking. For example, using recordings of children’s everyday speech, developmental psychologists can calculate two important indicators of intellectual functioning: the grammatical complexity of their sentences and the size of their working vocabularies. Why not do the same in schools? We could also employ a written version of this method, collecting random samples of children’s essays and stories for analysis.
Children are born wanting to find things out. But schools have, by and large, done little to build on this valuable impulse. In fact, when children get to school, they ask fewer questions, explore less often and with less intensity, and become less curious. One of the great ironies of our educational system is that it seems to squelch the impulse most essential to learning new things and to pursuing scientific discovery and invention. The good news is that researchers have developed excellent methods for measuring children’s interest in finding things out, as well as their ability to investigate in increasingly deliberate, thorough, and precise ways.
There are several ways we might measure a child’s disposition to inquire. We can easily record the number of questions the child asks during a given stretch of time. We can also rate those questions: Does the child ask questions that can be answered with data? Does the child persist in asking questions when he or she doesn’t get the answer right away? Does the child seem to use a range of techniques to get answers (such as asking someone else or manipulating objects)?
3. Flexible Thinking and the Use of Evidence
One of the most important capacities to be gained by going to school is the ability to think about a situation in several different ways. This has already been measured in college students. Why not measure it in younger children? Students could write essays in response to a prompt such as “Choose something you are good at and describe to your reader how you do it.” That would allow each student to draw on an area of expertise, assess his or her ability, describe a task logically, and convey real information and substance. A prompt of “Write a description of yourself from a friend’s (or enemy’s) point of view” would help gauge the ability to understand the perspectives of others, another invaluable skill.
Conversations are key to achieving many of the other goals here, but they’re also important in and of themselves. And they’re not hard to measure. Researchers have been analyzing conversations and the development of conversational skill for many years. Methods include looking at how long a conversation is (for example, how many sentences are uttered, how many words are used, how much time the conversation takes), how many turns each speaker takes, how many of these turns are in response to what was just said, how many topics are discussed, how full or deep the coverage of a topic is, and how attuned each speaker is to what has just been said. Outside coders could code children’s conversations for a number of characteristics: turns taken, depth of topic, amount of information exchanged, points of view articulated, and number of agreements and disagreements within the conversation. Analyses could also look at things such as the percentage of students in a given classroom who participate in conversations (to make sure that it’s not just one student or a small group doing all the talking). These analyses would have to take stock of what kinds of things children discuss and in what settings.
Children are born wanting to find things out. But schools have, by and large, done little to build on this valuable impulse. – Susan Engel
It would also be good to consider the role of the teacher in such conversations. Many studies have shown that adults play a crucial role in the acquisition of conversational skills. When researchers have recorded conversations between children and their parents at home, they have found that many parents talk frequently with their children, answering and asking questions, leading their children to expand and enrich their answers, and using the conversations to learn what their children are thinking about and what they know. Parents also use the conversations to offer their children new information about the world, as well as to teach (albeit without consciously trying to) the art of conversation. However, not all families are the same in this regard, and research has also shown that children living in poverty are much less likely to hear and be part of such rich exchanges at home. This makes it all the more essential for teachers to encourage a lot of discussion and verbal exchange at school.
It’s not always easy to help a child expand his or her linguistic or narrative repertoire, especially when the teacher and child come from different oral traditions. Teachers are given scant training about how to encourage, expand, and deepen children’s conversations. Schools of education offer lots of courses on curriculum planning, reading strategies, assessment, and classroom management, but I have seen few places where teachers deliberately reflect on or practice ways to have real conversations with their students.
If teachers knew that their students’ conversations were valuable and that they and their students were being measured by their conversations, they might get more help learning how to scaffold or enrich children’s talk. And unlike the kinds of “teaching to the test” we have come to know, which diminish a child’s educational experience, this kind of “teaching to the test” would improve children’s educational experiences day in and day out.
Vida had two young sons, both enrolled at the public school in her suburban community on the West Coast. Her older son, Quinn, was short, like his dad. But when you’re 9 years old, being shorter than the other boys is a liability. Quinn wore glasses for nearsightedness, and with his mom’s help he had chosen hip thick-rimmed glasses that had a band around the back to keep them in place; they made him look a little odd, almost as though he were wearing swimming goggles. He was a dreamer, happiest when he was lost in a book. He was reluctant to do sports and unsure of himself on the playground. He began to complain to his mother that he didn’t really have friends at school, and many mornings he didn’t want to go. Vida wasn’t sure how to help him. Then he began to tell her that lunch was the worst. A little boy named Sean, popular, athletic, and in command, had his own special table. All the kids referred to it as “Sean’s table,” and kids could only sit there by invitation. The children in Sean’s inner circle had permanent chairs at the table. Quinn wasn’t in the inner circle; he wasn’t even in the outer circle. Not knowing where to sit was making him miserable.
Though this kind of story appears again and again in parenting magazines and every group of parents has shared similar tales of social woe, the issue should be an educational one, not simply a parental ache. Teachers can help children like Quinn learn how to navigate their social settings, and helping children with this skill is surely just as valuable as teaching them to subtract and spell. But perhaps more important, teachers can help kids like Sean learn to resist the natural but undesirable impulse to exclude and dominate others in social settings. In order to do this, teachers need to devote time each day to guiding children through the jungle of social interaction.
One of the most robust findings in developmental psychology is that kids learn how to treat one another by watching the way adults treat them and treat each other. Yet few teacher-training programs emphasize the informal ways in which teachers behave. Nor do principals and superintendents attend much to how teachers treat children throughout the day or to how they interact with other teachers.
When parents ask me what to look for when visiting a new school, I always tell them to hang out in the hallways, looking at what is on the walls, listening to what teachers say to students as they pass by, and watching what teachers say to one another. If there’s one thing we know, it’s that collaboration and kindness emerge in a given setting only when such values permeate the group. The habits of kindness and teamwork need time, effort, and attention to develop. In other words, just as it’s important to assess whether children seem to be getting more skilled at helping each other and working together and are more inclined to do so, it’s important to assess the ways in which teachers are making such collaboration possible. It’s true that teachers might prep kids for such assessments, but in this case the prep itself might actually be of educational value.
To find out whether children are regularly absorbed in what they are learning, they need to be assessed in naturalistic settings. The important thing to find out is whether children are provided with opportunities to become fully absorbed in various kinds of activity. It is also essential to assess whether, given those opportunities, they concentrate on what they are doing and are energized by it.
The educational philosopher Harry Brighouse has suggested that the ability to think about something for 20 minutes at a time (sustained focus) may be one of the most powerful cognitive skills we acquire in school. Needless to say, some children seem to have great stores of such focus from the get-go while others find the road to sustained concentration long and nearly impossible to travel. But the measures I am arguing for here are not meant to show which child is better at concentrating and which child is worse. Instead, they are meant to ensure that the majority of children are meeting a basic benchmark. We don’t need to insist that children become ever more absorbed (in other words, children and schools don’t need to get higher marks each year). Nor should we hold back children who never seem as engaged as their buddies. Instead, the assessment should simply show that an individual child does become deeply immersed in one thing or another periodically. Similarly, the assessments can show whether a given classroom is providing enough opportunities for immersion. Thus the engagement measures, like others described here, provide proof of critical benchmarks for children and for classrooms.
I have argued that first and foremost children should be acquiring a sense of well-being in school. So why not ask them periodically how they feel? Questions might probe what they are working on that they care about, how often they like being there, whether they feel known by adults in the school, and how much of the time they feel interested in at least some of what they are doing. Economists and psychologists have shown that people are pretty reliable when it comes to telling us how happy they are. Why not use this metric in evaluating our schools?
These are just a few of the methods that would capture a child’s educational progress more effectively than the typical paper-and-pencil tests. Moreover, the metrics I’m proposing assess the capacities that actually matter in life outside of the testing room. We need an empirical snapshot of a school. By approaching assessment this way, we’d free up students and teachers to do more meaningful work.
Susan Engel is a developmental psychologist in the department of psychology at Williams College, where she is also the founder and director of the Williams Program in Teaching. Send comments to email@example.com.
(Copyright © 2015 by Susan Engel. This excerpt originally appeared in “End of the Rainbow: How Educating for Happiness (Not Money) Would Transform Our Schools,” published by The New Press.)