Book Review: Adler-Kassner and O’Neill’s Reframing Writing Assessment
Adler-Kassner, Linda and Peggy O’Neill. Reframing Writing Assessment to Improve Teaching and Learning. Logan: Utah State UP, 2010. Print.
Part scholarly monograph, part handbook, part rallying cry, Reframing Writing Assessment is an important addition to a spate of recent books on assessment that encourage K-12 and college teachers to take back our professional lives. (These books, like the one under review, tend to have titles featuring the prefix “re”: think Huot’s (Re)Articulating Writing Assessment, Maja Wilson’s Rethinking Rubrics in Writing Assessment, or my Reclaiming Assessment.) Specifically, Adler-Kassner and O’Neill’s goal is two-fold: “to convince readers—as individual professionals and as members of a profession—that we need to engage in […] discussions about writing pedagogy and assessment wherever they might occur” and to “provide readers with the necessary knowledge for engaging in these discussions” (9-10). The book is aimed at “all of us who care deeply and are invested in postsecondary writing instruction” (12). Though writing teachers and other interested parties might find the book illuminating, it is not a primer of writing assessment per se; in fact, it assumes readers already share a general disciplinary frame. The book will be most useful to Writing Program Administrators (WPAs) who share this frame and who are faced with the challenge of communicating about writing assessment with audiences within and beyond their institutions.
Indeed, the authors set the context and identify the exigency with the story of an anonymous WPA whose institution is considering adopting the Collegiate Learning Assessment (CLA). After consulting her colleagues on the WPA Listserv, she writes a memo to the university assessment committee outlining her concerns about the CLA. The committee decides to abandon the CLA, but this decision owes nothing to the WPA’s influence. In fact, she learns that an administrator called her memo “unhelpful” and “[n]one of her suggestions for a locally designed assessment that would track students across their undergraduate careers was adopted—nor was it even discussed seriously at the time” (2). While this is not a very uplifting story, the authors use it to underscore the need to get involved if we wish to influence writing assessment in our institutions.
Adler-Kassner and O’Neill’s recommended strategy for getting involved is “framing,” as in building construction. (Somewhat surprisingly, the authors do not cite Goffman’s sociological frame analysis, and Lakoff’s linguistic theory of framing makes a late appearance, on page 93). “Once we understand the frames already in place,” they write, “we can begin to develop ways to build on these structures, to reframe writing and writing assessment in a way that will support our values and beliefs” (11). To this end, Chapters 1-3 examine dominant frames for writing and writing assessment. Chapter 1 describes how arguments about these topics are nested in larger, values-based conversations and debates about the purposes of education. Chapter 2 extends this discussion by analyzing the currently commonsense frame for American education, in which the purpose of education is to prepare students for the twenty-first century workforce. This narrow frame has considerable support and force, but the authors remind us that postsecondary teachers still have a say in how and what to assess: under the 2008 Higher Education Act, institutions, in consultation with accreditors, set their own “standards and measures” for student learning (32). But this does not mean we should act only according to our own lights; on the contrary, Adler-Kassner and O’Neill gently chide compositionists for not doing enough to “effectively speak to questions about education and student learning asked by those outside the academy” (37; emphasis in original). They emphasize repeatedly that these questions are “entirely legitimate—that, in fact, publics outside the academy should understand what is happening in postsecondary classes and institutions” (38). If we want to exert some influence on writing assessment, the authors counsel, we must work with these others, not against them.
This means we must understand how historical frames have shaped and still shape writing and writing assessment, and so Chapter 3 broadens the historical lens, while narrowing the conceptual lens from education to composition and writing assessment. Drawing heavily on histories by Brereton, Connors, Crowley, and Elliott, the chapter elucidates twentieth century historical work through the authors’ “frame” analysis. For instance, the authors revisit the familiar story of the inauguration of the required first-year course at Harvard in order to show how the “technocratic frame” gave shape to writing instruction at this key historical moment and continues to exert influence today. This chapter also limns composition’s disciplinary frame, drawing from rhetorical, sociolinguistic, and literacy theory. This synoptic section is necessarily general and selective, but the authors do an admirable job identifying key insights and beliefs that circulate in the field: e.g., “textual meaning is bound by the context including the writer’s purpose and audience” (56); “[e]rrors are part of learning […] and are best addressed in context” (59); “good writing needs to be defined by what is appropriate and effective for a particular audience […] purpose […] and […] context” (64).
Disciplinary frames built around these insights and beliefs have little sway; for example, “even with composition’s enthusiastic promotion of portfolios as a way to reframe writing assessment, the dominant frames constructed in the late nineteenth and early twentieth centuries have persisted, especially when portfolio assessment is expanded to units of analysis beyond the individual classroom or writing program” (69). But Adler-Kassner and O’Neill are hopeful that writing assessment can help shift these larger frames, especially if the field links its frames to contemporary psychometric theory. They do not make a convincing case that the psychometric establishment has broken from its positivist roots, but they do show that some important thinkers in educational measurement are pushing validity and reliability theory toward the field’s ways of thinking. In any case, the authors end this chapter with a rousing if now-familiar call to “communicate our knowledge effectively beyond our own discipline and specialties so we can affect the discussions being held in state legislatures, departments of education, corporate boards, policy commissions, and public forums” (80).
Chapters 4-7 show how we can meet this challenge. Chapter 4 outlines “strategies and techniques” in a “frame-changing basics 101” format (83). These strategies and techniques, adopted from media strategists and community organizers, emphasize connecting our frames with those of other “stakeholders.” We are again reminded that “others outside of our classes and programs also have a heavy investment in what happens inside them” and “people genuinely care about what students learn in writing classes” (87). With this in mind, the authors explicate various models of alliance building. (Readers of Adler-Kassner’s excellent book The Activist WPA will find some overlap here.) While the review of these models helpfully offers an opportunity to think through how we position ourselves vis-à-vis others and their values, the authors emphasize the need to remain “within the larger frames surrounding the academy and education generally,” a prerequisite if we wish “to be understood as ‘legitimate’” (99). Accordingly, their rhetorical strategies and tips for communicating with stakeholders mostly entail accommodating audience expectations: e.g., “get to the point,” “check the media outlet’s guidelines,” and “be prepared and polite” (emphasis in original; 107-109).
Chapter 5 provides pseudonymous case studies of writing professionals framing and enacting writing assessment processes that meet institutional demands while drawing on their disciplinary expertise. The case studies are drawn from a range of institutions (a two-year college, comprehensive state university, Jesuit college, and technical university) and program types (writing, WAC, engineering programs, and a writing center). Adler-Kassner and O’Neill do not shy away from the difficulties of this work. They confront in each case the limitations of what has been accomplished: the alliance building is generally limited and targeted; one administrator admits that her institution still frames assessment as an administrative function rather than a disciplinary activity; another uses scoring guides developed by another program. While the authors’ insistence on providing realistic portraits in which readers can see themselves is admirable, the cumulative effect of these allowances is to make one wonder what exactly is being “reframed” in these examples, which, after all, involve mostly traditional assessment fare: outcomes, rubrics, conventional training and scoring protocols, etc.
Chapter 6 extends the authors’ consideration of “the messy complexities of reframing writing assessment,” and it features inspiring interviews with two compositionists who have participated in assessment discussions and initiatives at the national level. The pseudonymous Professors Chaco and Embler are worthy of emulation and they offer sound advice—“paint a sort of verbal portrait about the everyday realities of schools” (152), for instance, and “work across K-12 and college around assessment, while respecting each other’s experiences and expertise” (157). At the same time, Adler-Kassner and O’Neill’s accurate and compelling description of “accountability” as the dominant assessment frame will surely give pause to the aspiring Chacos and Emblers among us. Indeed, Professor Chaco informs us that “in the last ten to fifteen years, things have gone downhill in a frightening way because of the increasing reliance on assessments that are poorly conceived, and people don’t research, or don’t pay attention to research, on the negative impact [they are having in] schools” (154). The authors are led to admit that “others—such as policymakers, special interest groups, corporate CEOs—are constructing the frames that […] define teaching and learning” (166). However, Adler-Kassner and O’Neill are indefatigable: we can influence and change these frames, they insist, if we “engage them more directly” (166). This direct involvement, they emphasize again, requires that we “work with [the dominant frame]—even within it, if necessary” (178). The authors might like to take a “more radical stance,” but they “know such a positioning would fall on deaf ears and ultimately cause more harm than good” (177). They counsel against advancing assessment models that are “too far outside of this dominant frame,” lest they (and we) be viewed as “’unrealistic,’ ‘impractical,’ or worse” (177).
Chapter 7, the final chapter, pulls together the key strategies discussed throughout the book. It begins with a brief discussion of two metaphors—honeycombs and networked infrastructures—that the authors recommend as potential tools for “story-changing.” While the metaphors are interesting, they are involved, and this reviewer could have used more discussion of just how to mobilize them in conversations with various “stakeholders,” especially those outside of the academy. In any event, Adler-Kassner and O’Neil next provide a list of “essential tips” for reframing writing assessment such as “use rhetorical skills to construct frames and tell different stories,” “get involved,” and “value community-based work and support faculty who do it” (188-190). They end the book on a distinctly enthusiastic note: after listing national professional organizations, they entreat us to “check them out! And remember … reframing writing assessment, like writing, is a process that improves with revision and practice” (191).
Readers are meant to close the book confident that we’ve been provided the encouragement and resources necessary to get involved and that our involvement will make a difference. Some readers, though, will have some nagging questions: Is it the case that all “outside others” have “legitimate concerns” and “genuinely care” about our students and their learning? Is it true that if we learn to argue better, people in power will start listening to us? What if the problem faced by the WPA at the beginning of the book wasn’t that she wrote an unconvincing memo but that no one read it in the first place? Similarly, what if the problem we face as a field isn’t that we’ve been unclear about the value of portfolios but that (as Professor Chaco implies) no one is listening to us? After all, frames are constructed as much to exclude as to include. The “policymakers, special interest groups, [and] corporate CEOs” that Adler-Kassner and O’Neil rightly say control the dominant frames for education are all too happy to exclude and—as we’ve seen in recent events in Wisconsin—even demonize the frames (and the working bodies) of teachers and program administrators. So while surely we should get involved in discussions about writing assessment, we need to anticipate the possibility that even our very best arguments from our most careful self-positionings may not find willing ears. Our involvement in discussions about assessment might need to entail intervention into how they work, and who gets to speak, in the first place. Reasonable, moderate, cooperative participation—a seat at the “stakeholders” table—may not be enough.
Still, Reframing Writing Assessment will help us be and talk smarter about writing assessment. This alone might not be sufficient to take back our professional lives, but it is necessary.
- Adler-Kassner, Linda. The Activist WPA: Changing Stories about Writing and Writers. Logan: Utah State UP, 2008. Print.
- Gallagher, Chris W. Reclaiming Assessment: A Better Alternative to the Accountability Agenda. Portsmouth, NH: Heinemann, 2007. Print.
- Huot, Brian. (Re)Articulating Writing Assessment for Teaching and Learning. Logan: Utah State UP, 2002. Print.
- Wilson, Maja. Rethinking Rubrics in Writing Assessment. Portsmouth, NH: Heinemann, 2006. Print.
Chris W. Gallagher is Professor of English and Writing Program Director at Northeastern University, where he teaches writing and rhetoric courses. He is the author of three books, including Reclaiming Assessment, and numerous articles on writing pedagogy and assessment as well as accountability and educational reform.