Assessment

The Georgetown University Writing Program regularly evaluates its effectiveness by conducting research into how Georgetown faculty approach writing in their courses, students’ views of writing, and the writing that students produce. We began in 2013 with a benchmark study, in which Writing faculty scored sample student papers using a rubric reflecting the learning goals for first-year writing. Scoring showed that students were competent at addressing the purpose and context of their writing tasks and in organizing their essays, but many relied on formulaic approaches to argument and support and used an overly-formal, often convoluted style. Findings were shared with faculty, who found them neither surprising nor helpful. However, the faculty who participated in this study noted that they gained useful insights from talking with each other as they read the papers. This led us to emphasize forms of assessment that facilitate faculty conversations. 

Between 2015 and 2017, the Program conducted an institutional ethnography, combining surveys and an analysis of syllabi from our first-year writing course with focus groups with faculty and students to explore how they viewed writing instruction, how they evaluated students’ writing, and how their engagements with writing were shaped by institutional conditions. The Program collected similar evidence for Integrated Writing and considered data from surveys of current students and alumni conducted by the Office of Assessment and Decision Support. Across the board, these studies revealed that faculty and students alike value writing and the first-year writing course, but they also pointed us to some distinctly different expectations. Some faculty and students view writing as a complex, iterative or developmental intellectual and creative task, while others understand writing more narrowly in terms of students’ ability to construct clear sentences and organize content. Research also found that while faculty talk about rhetorical concerns such as audience and genre, these concepts were not often visible in writing syllabi or assignments, either in first-year courses or in the disciplines. Put simply, critical, complex thinking about writing was not playing out as explicitly as we hoped. This shaped a series of discussions among first-year writing faculty and faculty in other disciplines about how people develop as writers, the well-documented link between writers’ developing understanding of a topic and the clarity of their prose, differences in how arguments are framed and supported across the disciplines, and the value of approaching writing as a process of thinking about content, situation, and genre. Based on these discussions, we revised the learning goals for WRIT015 in 2018 to more clearly emphasize core ideas about writing, and our faculty have worked together to explore ways of teaching these ideas and how students apply them.

In 2019, the Writing Program began using the “dynamic criteria mapping” model of assessment developed by Bob Broad (2009). In two rounds, samples of student writing have been discussed in faculty focus groups, along with the assignment prompts for these papers. About one third of first-year writing faculty participated in this process, with papers drawn from more than 80% of sections. The Assessment Committee coded these transcripts and generated concept maps that articulate central themes and position them in relation to each other. Writing faculty discussed the implications of these maps in meetings in Fall 2021, “closing the loop” by identifying questions and themes for shared faculty work over the next year. The Assessment Committee has noted several strengths of this process. First, the process itself generates meaningful conversations and opportunities for shared learning. Repeatedly, faculty have commented — during the focus groups and after — on insights they’ve gained from reading and discussing work produced in their colleagues’ classes. Second, building on a well-established tradition of faculty conversations about teaching writing, the maps should facilitate critical and reflective discussions of what the Program is achieving and challenges that are shared across sections.