Closing Deals with Hamlet's Help: Assessing the Instrumental Value of an English Degree

Closing Deals with Hamlet’s Help 357

ecent decades have seen a number of declensionist studies of college students. Books such as Naomi Riley’s The Faculty Lounges: And Other Reasons Why You Won’t Get the College Education You Paid For and Andrew Hacker and Claudia Dreifus’s Higher Education? How Colleges Are Wasting Our Money and Failing

Our Kids—And What We Can Do about It have taken up the mantle of educational decline, arguing, as Hacker and Dreifus do, that “higher education has lost track of its original and enduring purpose” (9). This vague sense that things have gone off track—that students and faculty somehow used to be better—is a key driver of the most recent turn toward assessment in higher education.1

As Mary Allen correctly notes, accrediting agencies have been “the major force promoting the use of assessment” in higher education (2). They are the chief promoters of an “increased emphasis on student learning,” at least insofar as such learning can be measured and monitored by “empirical data” (1–2). Of course, the need for such “increased emphasis” rests on the assumption that student learning has been, prior to the move toward assessment, insufficiently emphasized. The assessment turn, in other words, is a symptom of the declensionist turn. It rests on a single, simple assumption: that higher education has somehow lost its way and needs to be set back on course.

College English, Volume 76, Number 4, March 2014

Closing Deals with Hamlet’s Help: Assessing the Instrumental Value of an English Degree

R

Sheryl I . Fontaine and Stephen J. Mexal

Sheryl I . Fontaine is a professor of English and Interim Dean for the College of Humanities and Social Sciences at California State University–Fullerton. Her publications include articles on pedagogy and the development of the profession and have appeared in CCC, JAC, and ADE Bulletin, as well as in books coedited with Susan Hunter: Unheard Voices in Composition Studies (Southern Illinois UP 1993), Foregrounding Ethical Awareness in Composition Studies (Boynton-Cook, 1998) and Collaborative Writing in Composition Studies (Wadsworth, 2006); with Pat Belanoff and Peter Elbow: Nothing Begins with N: New Investigations of Freewriting (Southern Illinois UP, 1991); with Pat Belanoff, Marcia Dickson, and Charles Moran: Writing with Elbow (Utah State UP, 2002); and coauthored with Cherryl Smith: Writing Your Way through College (Heinemann, 2008). With Steve Westbrook, she currently coedits the series Lenses on Composition Studies for Parlor Press. Stephen J. Mexal is associate professor of English at California State University–Fullerton. His essays have been published in MELUS, English Language Notes, Studies in the Novel, ADE Bulletin, Inside Higher Ed, and the Chronicle Review. He is the author of Reading for Liberalism: The Overland Monthly and the Writing of the Modern American West (U of Nebraska P, 2013).

J357-378-Mar14-CE.indd 357 1/20/14 3:54 PM

358 College English

Drawing on our own research and what we judge to be a neglected means of as-sessment, we are questioning the assumption that higher education is in decline. We are interested in education not as a sort of language game to be played and measured within the walls of the university, but rather as something with potentially long-lasting effects. We wanted to know if our students perceive a meaningful connection between their college English classes and their postcollege professional lives, years and even decades after leaving the classroom. Using what we identify as a retrospective indirect assessment tool—one based on self-reported perceptions of learning from former students—we surveyed fifty years of English alumni from our large, master’s-comprehensive public university. Valuing the subjective, accretive nature of education, we asked respondents to assess the multivariate links between our English curriculum and their subsequent employment. What we learned from our former students rebuts common assumptions about the decline of student learning and the related critiques of higher education.

One recent and highly cited critique of student learning and higher education is Richard Arum and Josipa Roksa’s 2011 book Academically Adrift. Arum and Roksa reach their conclusions by sifting through results of the Collegiate Learning Assess-ment (CLA), a popular example of the kind of direct assessments produced by the declensionist-driven turn. The CLA is a standardized test first given in 2000 that makes use of the growing interest in a “value-added” approach to assessing student learning. Like other such tests, the CLA attempts to measure not just absolute knowledge of a certain subject or subjects, but the improvement in specific skills that are assumed to transcend disciplines, charting the “value” that has been “added” as a result of a student’s college education. By purporting to assess student improvement during the first three years of college in four areas—critical thinking, analytical reasoning, problem solving, and writing—the CLA is designed to measure students’ academic preparation as well as the knowledge and skills imparted by higher learning.

While debate continues about the validity and execution of the CLA itself (see Glenn, Hosch, Possin, Sternberg, and Swail), these conversations have not yet di-minished its use. Nor have they silenced the alarms sounded by critics who claim that the results of this study of student learning are, in the time-honored tradition of such things, not good.

According to Arum and Roksa’s reading of the CLA, for four-year undergradu-ate colleges and universities, “gains in student performance are disturbingly low,” and although student learning is low overall, the authors also caution that there is “notable variation both within and across institutions that is associated with measurable dif-ferences in students’ educational experiences” (30). They claim, in other words, that modern universities do little for their students, who exist in a kind of mirror image of Garrison Keillor’s Lake Wobegon, where all students are below average—and students from certain schools and disciplines are even less average than others.

Several years after these results were first published, they remain the subject of

J357-378-Mar14-CE.indd 358 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 359

numerous articles, commentaries, blog posts, and, undoubtedly, faculty meetings and retreats in which the value of college education is doubted, faculty devotion to un-dergraduate education questioned, and students’ academic worthiness distrusted (see Attewell 225-26; Bauerlein 352–58; Bérubé 364–66; Caspers 47-49; Chesbrough 2-8; Goral 20; Igo 53-56; Menand 74–79; Overman 129; “Reviews of Academically Adrift” 487–90). In one of the more cynical analyses, Kevin Carey concluded after reading the book that “there are two kinds of college students in America”:

A minority of them start with a good high-school education and attend colleges that challenge them with hard work. They learn some things worth knowing. The rest—most college students—start underprepared, and go to colleges that ask little of them and provide little in return. Their gains are minimal or nonexistent. Among them, those with reasonable facility for getting out of bed in the morning and navi-gating a bureaucracy receive a credential that falsely certifies learning. Others don’t get even that.

To a certain degree, such complaints about college students are timeless. (As far back as 1907, the president of the University of Wisconsin was bemoaning the increasing number of students “of no very serious purpose” [Rudolph 290].) After all, declen-sionism is evergreen. Things are never as good as they were in the good old days.

But educational declensionism specifically is often driven by expanded access to higher education. One of the earliest moves designed to allow more student access to higher education was in 1862, when President Lincoln signed the Morrill Act into law. It granted federal land to states, leading to the institution of land-grant colleges and universities and the establishment of college programs in the “useful arts,” including agriculture, mechanics, mining, and military instruction (Thelin 76). If elite, private universities were charged with providing the cultural and professional finishing of their patrician students, land-grant colleges were designed to accom-modate the rest of the student population and accordingly to have an instrumental value. Instrumental, in this sense, refers to an applied or practical use value. Land-grant colleges were intended to have a practical or instrumental utility. That model of education contrasted the then-dominant liberal arts model, which, in its classical form, is noninstrumental. Students in the liberal arts tradition study art, mathematics, music, and so on for those subjects’ own sakes, not for any practical or applied—that is, instrumental—value they might have.

But the Morrill Act specified that certain college programs should teach “useful arts” with instrumental outcomes esteemed for their technological, economic, or so-cial applications, including subjects such as agriculture. Agriculture and mining—like a host of other subjects that characterize the modern large university, including nurs-ing and business—are noninstrumental academic disciplines that are also esteemed for their instrumental value: for their technological, economic, or social applications. Although subjects like these continue to be valued for their potential application,

J357-378-Mar14-CE.indd 359 1/20/14 3:54 PM

360 College English

they have since gained value as noninstrumental academic disciplines with bodies of knowledge to be researched and theorized for their own sake.

In the age of assessment, the relationship between instrumental and nonin-strumental knowledge has produced a conflict. Assessment is, in its assumptions, instrumentalist. It assumes that learning has occurred only insofar as that learning can be applied and measured, even if the only “application” is on an examination. Assessment takes as an article of faith that anything of value can be measured, and that only things that can be measured can have value. Yet assessment strategies fre-quently have to take stock of noninstrumentalist learning, in which knowledge does not have a direct and designed relationship with any particular extra-academic utility. But even so, the ultimate justification for assessment is the instrumentalist conten-tion that student learning is for something: that it will eventually be applied to some extra-academic purpose, such as economic, technological, or social improvement.

It is at this nexus—the meeting point of college assessment, instrumental learn-ing, and value as determined by the marketplace—that our study seeks to contribute. By surveying English alumni for their perceptions of the connections between their college studies in English and their current field of employment, we provide one way to weld the noninstrumentalist substance of assessment procedures with the instrumentalist underpinning of those same procedures.

T h e C L A , N o N i N s T r u m e N T A L i s T A s s e s s m e N T , A N d T h e P r o b L e m w i T h P u r P o s e

The 500 or so universities that administer the CLA are, by and large, public universi-ties known more for their accessibility than their exclusivity. These are institutions of higher education that, since the Morrill Act, have been charged with instilling in their students useful preparation for careers in agriculture, engineering, journalism, social work, education, and other fields. In contrast, administrators of the CLA are looking at largely noninstrumental skills of critical thinking, analytical reasoning, problem solving, and writing: that is, skills not actively directed toward a particular economic or social utility.

Early in their book, Arum and Roksa briefly allude to this issue of utility. In their first chapter, they frame their study as a consequence of an expanding set of concerns about higher education. They claim legislators and parents worry over the “returns to their investments” in college; they note that businesses worry “whether graduates have acquired the necessary skills to ensure economic competitiveness”; and they allude to the “widespread agreement” that student “capacity for critical thinking and complex reasoning” is necessary in this “rapidly changing economy and society” (1–2).

J357-378-Mar14-CE.indd 360 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 361

But after this early instrumentalist justification—in which they explain the need to assess student learning by appealing to the ultimate social and economic utility of that learning—they seem to forget about the possibility that student learning might not be merely academic. By the book’s second chapter, they have abandoned any extracurricular justification and instead simply refer to an academic (rather than a social or economic) consensus about the importance of critical thinking and effective communication (Arum and Roksa 35). They begin with an open system, asserting the connections between classroom learning and the “rapidly changing economy,” but quickly transition into a closed system, defending the value of learning certain things simply because “99 percent of college faculty” say those things are important (2, 35). Of course, as their invocation of a faculty majority suggests, professional consensus does matter, and the sum value of a college degree cannot be measured by the job in which you find yourself after graduation.

However, Arum and Roksa and their supporters want to have it both ways. Aca-demically Adrift makes the case that US colleges and universities, and their students, are—wait for it—“adrift.” Drift shares an etymological root with drive; in its earliest usage, it refers to the act or action of driving. It suggests a controlling agent: a per-son or thing actively propelling another person or thing. Adrift, in contrast, implies the absence of such an agent. The word came into use during the sixteenth-century explosion in global exploration and ocean trafficking. Unlike drift, adrift has always been a nautical term suggesting a deliberative human agency. One can, after all, be drifted by the tides yet still adrift if the captain is not steering the vessel. To be adrift is not to be undriven, but rather to be driven without purpose.

So Arum and Roksa want to critique schools and students for being adrift, or not being deliberatively guided toward a particular destination, yet at the same time they restrict their analysis of student improvement to a limited set of skills that are performed while students are in college. Such an analysis either ignores instrumen-talist justifications that insist learning is worthwhile because it leads students to a particular destination, or it assumes, with no evidence, the connection between these college skills and a student’s destination. To use their own metaphor, Arum and Roksa want sailors to, say, become skilled at jumping the halyard to raise the mainsail, but then only to demonstrate this skill while the ship tacks back and forth in the harbor.

However, even if the authors believe that acquiring this skill will be useful for getting the ship to Portofino all the faster—that is, that noninstrumental academic knowledge might one day acquire an instrumental, applied dimension—they don’t really know if that’s the case. Mostly they want students to acquire noninstrumental knowledge just because everyone seems to think it’s worth doing. But doing some-thing because everyone thinks it’s a good idea offers no guarantees against being adrift. On the contrary, when sailors acquire skills with their instrumental utility in mind is when the ship is most likely to sail aimfully.

J357-378-Mar14-CE.indd 361 1/20/14 3:54 PM

362 College English

Of course, the metaphor is imperfect. Liberal education is not the same thing as learning to work the running rigging. And though it is telling that Arum and Roksa use an instrumentalist nautical metaphor to frame their largely noninstrumentalist view of higher education, their book should not be allowed to founder on a single figure of speech.

Yet in the current debates about assessment, critics and faculty alike frequently overlook an important dimension of the assessment of student learning: namely, the social and economic consequences used to justify the assessment measures in the first place. Many critics skirt this consequentialist line of thought because measuring socioeconomic consequences, at least when it comes to liberal education, is tricky stuff. They are afraid or unwilling to follow the ship out of the harbor.

Unlike liberal higher education, the socioeconomic utility of vocational educa-tion is relatively straightforward. Heating, ventilation, and air conditioning (HVAC) certification, for instance, is instrumental learning: it has a direct and designed relationship with consequent employment in the field of HVAC repair. But higher education in the liberal arts and humanities by its very nature has no direct and de-signed relationship with consequent employment in any particular field. Vocational education and employment are bound by a firm, bidirectional relationship. It is rea-sonable that an HVAC repairperson has studied HVAC repair, but the converse is also true: it is reasonable that a person studying HVAC repair is doing so to become an HVAC repairperson. In contrast, liberal education and employment involves a tenuous, unidirectional relationship. It is reasonable that an HVAC repairperson has an undergraduate degree in philosophy, but here the converse is not just untrue but ridiculous: it is thoroughly unreasonable that a person pursuing an undergraduate degree in philosophy is doing so to become an HVAC repairperson.

However, it is equally untrue that undergraduate education has no bearing what-soever on students’ postcollege professional lives. This is the instrumentalist view of student learning assumed, and yet so quickly forgotten, by assessment advocates. If it is important to assess student learning in college, it is important not just for local or particularistic reasons, but for instrumental reasons as well. We want students to learn in part because we assume that their learning sails beyond the ivory tower, that it has applied consequences for the civic, economic, technological, or moral health of society.

Meaningful student assessment must, in other words, take into account the un-intended, transferable utility of liberal higher education. If educators want to assess student learning, we should drop the pretense that college learning is a completely closed system, a sort of language game subject only to its own particular rules, norms, and practices. We cannot simultaneously claim to assess student learning solely as a good in itself, something that is supposed to be noninstrumental, while also claim-ing that student learning has instrumental consequences for the world outside the university, however indirect and undesigned they may be.

J357-378-Mar14-CE.indd 362 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 363

Among other findings, Arum and Roksa report that students from the humani-ties, social sciences, hard sciences, and math scored better on the CLA than their counterparts in business, education, and social work. In a surprising contrast to what earlier generations of academics from land-grant or public universities might have anticipated, the more overtly “useful” majors—the applied or instrumentalist-leaning majors—showed less improvement on the skills studied by the CLA than did the degrees lacking a clear instrumentalist component. Arum and Roksa explain this difference by reasoning that students in the noninstrumentalist majors wrote and studied more. So after controlling for students’ study patterns, social backgrounds, academic preparation, and the amount and type of writing and reading assignments, they found a “substantial” decrease in the “gaps between science/mathematics and business majors as well as between social studies/humanities and business majors” (106).

In other words, Arum and Roksa swept disciplinary differences under a rug of statistical analysis without ever looking at the potential value those differences might provide to conversations about the value of a college education. While it is one thing to claim that research should factor out students’ social backgrounds, it is quite an-other to factor out qualities that are in fact essential to the nature of the disciplines.

To factor out the study patterns and types of assignments that characterize a field of study is, put simply, to take the discipline out of the discipline. What is a discipline other than the dispositions that characterize its research and conversation, its texts and scholarship—the very things that form the basis for the study patterns and assignments that are apparent in our curricula and shape the academic maturity of our students? To strip from students in the humanities and social sciences their study patterns and the kind of writing they perform strips them of what study in those fields provides. Rather than removing these skills from the equation, we should be asking what value they have added to students’ education.

In addition to looking at disciplinary distinctions, colleges and departments also need to determine ways of expanding their appraisals of students in order to assess the transferability of these discipline-based skills beyond the college walls. Tests that peer inside college classrooms to see the “value” that those classes have added to stu-dents’ thinking and writing may indicate something about gains in noninstrumental skills, but they are not much better than Ouija boards at accurately predicting the ultimate course of students’ lives. The CLA, like any other direct assessment mea-sure, does a poor job anticipating the waypoints that mark students’ journeys from the classroom to their professional lives. As the authors of a recent study released by Stanford University suggest, the “long-term value of an education is to be found not merely in the accumulation of knowledge or skills but in the capacity to forge fresh connections between them, to integrate different elements from one’s education and experience and bring them to bear on new challenges and problems” (Auletta). But

J357-378-Mar14-CE.indd 363 1/20/14 3:54 PM

364 College English

if this is true, then how best to measure students’ capacity to forge such connections on “new challenges and problems,” particularly those they face after graduation?

The course between university and career can be easily misread. Consider the case of Damon Horowitz, who has a PhD in philosophy and currently serves as “in-house philosopher” at Google (Horowitz). In 2011, Horowitz gave a presenta-tion to students at Stanford on why, as the Chronicle of Higher Education later put it, “you should quit your technology job and get a Ph.D. in the humanities.” Horowitz provides an eloquent narrative to describe what appears to be an unexpected path from artificial intelligence (AI) to philosophy:

[T]he questions I was asking [in AI] were philosophical questions—about the nature of thought, the structure of language, the grounds of meaning. So if I really hoped to make major progress in AI, the best place to do this wouldn’t be another AI lab. If I really wanted to build a better thinker, I should go study philosophy.

Of course, Horowitz is telling a story here. Like all historians and biographers, he is retroactively tracing cause and effect. Because he “hoped to make major prog-ress in AI,” he says, he chose to “go study philosophy.” First was the desire for an instrumental benefit, then the education to fulfill that desire. And it is possible that Horowitz’s life was just as linear as he makes it sound. But somehow we doubt it. For most people, after all, the comforting linearity of biography only emerges in retrospect. In other words, the translation from noninstrumental academic learning to instrumental professional skills is unique to each person, and nearly impossible to foresee with any clarity. We like to think that we begin on the shore and confidently plan our journey. But what really happens is that one day we walk to the stern and gaze backward, marveling at how far we’ve come and squinting to recall the course, now grown hazy with age and distance, that we did not know we were charting at the time.

Although career centers on college campuses have long distributed exit surveys to graduates asking for their employment reports, the use of such information is limited, often becoming brag sheets of companies and professions meant either to reflect well on the college or, alternatively, to lament of the plight of the recent graduate. If we want to know something about the paths our students take, the point from which we should be mapping their course lies far beyond the shoreline of our classrooms, even farther out than the one marked by our colleagues in the career center. Tak-ing a cue from Horowitz, then, and believing that disciplinary differences may help us to understand the navigational courses that emerge as seemingly nonlinear and unpredictable paths from the college degree to the life after college, we surveyed the alumni of the Department of English, Comparative Literature, and Linguistics at California State University–Fullerton.

J357-378-Mar14-CE.indd 364 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 365

i N s T i T u T i o N A L d A T A A N d s u r v e y r e s u L T s

California State University–Fullerton (CSUF) is among the largest universities in the twenty-three-campus California State University system, with a student body of 36,156 in 2011. Although not technically a land-grant college, CSUF, classified by the Carnegie Foundation as a “Master’s L” institution (that is, a large master’s-comprehensive college or university), can nonetheless be thought of as continuing the historical mission of those schools. CSUF awards a significant number of degrees to minority and first-generation college students. (It is currently fourth in the nation in bachelor’s degrees awarded to Hispanics and eighth in degrees awarded to minority students overall [CSUF 2012 Fact Sheet]). The Department of English, Comparative Literature, and Linguistics at CSUF is one of twenty departments and programs in the College of Humanities and Social Sciences. In 2010, the college awarded 1,752 bachelor’s degrees, more than any other college in the California State system, including the Mihaylo College of Business and Economics (which awarded 1,738). The English department awarded 154 bachelor’s degrees in the 2010 academic year, the fourth largest number of degrees awarded in the college.

We provide this detailed profile to highlight that ours is a nonselective, regional public university and, in that way, is fairly representative of many public universities in the country. It is unsurprising that humanities majors from elite universities often go on to attain prestigious or remunerative careers, but when people talk about em-ployment prospects for humanities students, they are usually talking not about schools like Cornell and Stanford, but about schools like CSUF, and about students like ours.

While we are undoubtedly committed to developing the academic success of our students and supporting the scholarly agenda of our faculty, we are also aware that as a regional public university, we are expected to be both accessible and accountable to the public and the professional future of our students. There has long been a push toward what is variously called experiential, professional, skills-based, and occasion-ally vocational knowledge—learning to gain transferable, applicable skills rather than pure knowledge—but the push is a soft one, and frequently met by an opposing push toward nurturing the intellectual growth of the mind.

Since its founding over fifty years ago, the CSUF Department of English, Comparative Literature, and Linguistics has not, either by design or by disciplinary circumstance, had a particularly skills-based or professionally oriented curriculum. Other than the courses for our secondary credential and a tutor-training course, the undergraduate curriculum has remained more or less as it was at its inception: a familiar distribution of courses across literary genres, periods, authors, and theories. Aside from the inevitable expansion of authors, the stretching of periods, and the introduction of new fields such as composition and rhetoric in the 1980s, the department curricula

J357-378-Mar14-CE.indd 365 1/20/14 3:54 PM

366 College English

have remained stable. Indeed, we imagine that most school like ours, with student populations like ours, have similarly configured and stable curricula.

But like its land-grant brethren, our university tries to accentuate the practical, instrumentalist value of learning alongside the acquisition and production of pure knowledge. Our university mission statement asserts that the school strives to “pre-pare our diverse student body for academic and professional success,” and that faculty, staff, and students “work in close collaboration, fostering personal and professional development and advancing knowledge” (California State University Fullerton Integrated Strategic Plan; emphasis added).

In that spirit, we surveyed our alumni in order to better discern the connection between students’ English education and their current careers. We sought to discover how, if at all, former students had extracted instrumental value from their noninstru-mental liberal arts education in college. If our students are acquiring certain skills from studying, say, Shakespeare, what are those skills, and what is their use value in the workplace? Although we have analyzed other components of the survey elsewhere and did include some questions in the survey that encouraged students to extract from their academic studies any number of applied values—civic, ethical, social, moral, or virtually anything else—for this particular report, we focused on our respondents’ perceived applications of their education in their professional lives (Fontaine and Mexal). We simply wanted to ask alumni to examine their subjective senses of the applied professional value of their English education, many years after that learning has occurred and they have had ample time to extract instrumental utility from their noninstrumental academic studies.

Since the assessment turn, the effort to translate noninstrumental learning into measurable skills has produced a lot of fuzzy language: superficially straightforward con-cepts (that nonetheless lack widely agreed-upon meanings) such as “critical thinking” or “numerical literacy.” But rather than pretend that critical thinking is an uncomplex, widely accepted thing (like, say, the difference between an inch and a centimeter), we wanted to acknowledge the truth: that it is a highly subjective variable in which two reasonable people could plausibly see two very different things (like, say, the difference between the colors orange and red). In response to this dilemma, we simply deferred to the respondents’ subjective sense of their skills in the quantitative portion of the survey and provided space, in the qualitative portion, for them to write the narrative details from which the categories of our qualitative analysis would later emerge.

Many assessment advocates assert that indirect assessment is “less clear and less convincing” (Suskie 20) than direct assessment; it can be “subjective, biased,” and “may not provide accurate information” because it focuses on a student’s individual perceptions of his or her learning (Lee 57). But learning is subjective and individual. And because indirect assessment recognizes the importance of both of these qualities, it is potentially among the most valid and valuable means of assessing the success of

J357-378-Mar14-CE.indd 366 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 367

student learning. We are certainly not alone in arguing that indirect measures should be included, with empirical qualitative tools, as an important component of evidence collection (see Allen; Maki; Hogan, Lusher and Mondal; Menhart; Yamayee and Albright).

Administered by our campus’s Social Science Research Center, the study surveyed English department alumni as well as current graduating seniors. Because we are a relatively new university, founded in 1957, we decided to attempt a comprehensive study of alumni since 1960, when the first alumni data were recorded. We wanted the survey to be particular to our students and program, so we custom designed questions that invoked the classes, programs, and support structures in our department and at our university.

Using email as well as follow-up postcards, we achieved an impressively high response rate of 26.2% for alumni (n = 682), which was, on balance, negatively cor-related to time since graduation. In other words, our smallest response rate was among those graduating between 1960 and 1969, the alumni whose contact information was the oldest. With this imbalance in mind, we analyzed our data by decade in order to discern any statistical differences among the groups. Ultimately, we found no statistical difference among those who graduated in these years: 1960–69, 1970–79, 1980–89, 1990–99, 2000–09, and 2010–11.

Overall, respondents assessed their perceived gains in English-associated skill sets positively: 84% felt they had improved their written communication skills “a great deal” (46.8%) or “quite a bit” (37.2%) as a result of their education at CSUF. Similar figures were reported for gains in knowledge of English and American literary tradi-tions (44.4% reported “a great deal” of improvement; 39.4% reported “quite a bit”), analytic reading skills (42% reported “a great deal of improvement”; 42.3% reported “quite a bit”), and other associated areas. (See Table 1.)

This indirect, retrospective skills assessment foregrounds the clarity of hindsight. Like Horowitz, Google’s in-house philosopher, we can best comprehend the last-ing significance of educational experiences only well after they have occurred. Such confirmation of students’ learning also explains respondents’ subsequent endorsement of their English degree as a positive factor in their professional lives. There was a strong positive correlation between skills cultivated by the study of English and the reported importance of those skills in the workplace. Respondents who felt their reading skills improved when they were students were also likely to feel that their analytic reading skills were important in their workplaces now.

We also studied the degree to which respondents felt their studies at CSUF cultivated particular skills and then correlated that data set with the degree to which respondents felt those skills were important in their professional lives: 83.8% reported that their written communication skills improved “a great deal” (46.8%) or “quite a bit” (37.2%) as a result of studying English at CSUF, and 95% felt those skills were

J357-378-Mar14-CE.indd 367 1/20/14 3:54 PM

368 College English

“extremely” (72.1%) or “very important” (22.9%) in their professional lives. In ad-dition, 83.6% reported their analytic reading skills improved “a great deal” (42.3%) or “quite a bit” (41.3%) as a result of studying English, and 88.7% reported those same skills were “extremely” (52.6%) or “very important” (36.1%) in their work-places. And 78.3% thought their ability to find and evaluate information improved “a great deal” (40.3%) or “quite a bit” (38%) at CSUF; 92.4% reported that skill to be “extremely” (60.9%) or “very important” (31.5%) in their professional lives. Similarly, 60.7% reported their knowledge of grammar and syntax improved “a great deal” (26.1%) or “quite a bit” (34.6%) as a result of studying English at CSUF, and 83.9% of respondents thought such knowledge was “extremely” (50.9%) or “very important” (33.0%) in their professional lives. (See Table 2.) There was, in sum, a significant correlation between the respondents’ perceived educational gains in par-ticular skill areas and the perceived professional importance of those same skill areas.

However, although we were able to determine that there was a significant correlation between respondents’ perception of the skills they had honed through

Content Area“From your experiences studying English, Comparative Literature [or] Linguistics at CSUF, how much do you feel that you gained in each of these areas?”

“Nothing at All” (%)

“A Little” (%)

“A Moderate Amount” (%)

“Quite a Bit” (%)

“A Great Deal” (%)

Written Communication Skills 0.4 2.5 13.1 37.2 46.8

Knowledge of English and American Literary Traditions

0.8 1.5 13.9 39.4 44.4

Analytic Reading Skills 0.5 2.9 13.0 41.3 42.3

The Ability to Learn Independently 1.1 3.4 18.0 36.4 41.1

The Ability to Find and Evaluate Information

0.5 2.8 18.4 38.0 40.3

The Ability to Consider Things from Others’ Perspectives

0.4 3.3 17.1 40.0 39.2

The Ability to Entertain Multiple Conflicting Views

0.5 4.0 18.3 40.3 36.9

Logical Thinking 0.6 4.2 19.7 41.2 34.3

Creativity and Imagination 0.9 6.5 22.4 37.4 32.8

Knowledge of Grammar and Syntax 1.6 11.3 26.4 34.6 26.1

Knowledge of Global Literary Traditions 2.8 15.3 30.5 32.4 19.0

Table 1. Skills Improvement as a Consequence of Studying English

J357-378-Mar14-CE.indd 368 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 369

higher education and the subsequent significance of those skills in their professional lives, the multivariate nature of that correlation was more difficult to discern, at least through quantitative measures.

For example, 83.8% of respondents reported “a great deal” (44.4%) or “quite a bit” (39.4%) of improvement in their knowledge of English and American liter-ary traditions. This is not particularly surprising; our curriculum has been a fairly traditional literary-historical model for decades. But what is surprising is that 77.6% of respondents reported that such knowledge was “somewhat” (26.8%), “very” (26.7%) or “extremely important” (24.1%) in their professions. This figure cannot be explained by a purely instrumentalist view of such knowledge: only about 45% of respondents reported working as K–12 teachers or as college instructors or profes-sors. This means that there was a not-insignificant percentage of respondents who perceived instrumental or vocational value in noninstrumental, wholly academic learning. There are a significant number of English department alumni who are not employed to read, teach, or think about, say, Shakespeare, who nonetheless have

Content Area“How important have you found each of the following to be in your professional life?”

“Not at all Important” (%)

“Not Very Important” (%)

“Somewhat Important” (%)

“Very Important” (%)

“Extremely Important” (%)

Written Communication Skills 0.7 0.6 3.7 22.9 72.1

The Ability to Learn Independently

0.6 1.5 5.7 31.3 61.0

Logical Thinking 0.3 0.1 6.8 31.8 60.9

The Ability to Find and Evaluate Information

0.1 0.6 6.9 31.5 60.9

The Ability to Consider Things from Others’ Perspectives

0.7 1.5 7.1 31.6 59.0

Creativity and Imagination 1.5 2.8 10.7 31.2 53.8

Analytic Reading Skills 1.3 1.6 8.3 36.1 52.6

Knowledge of Syntax and Grammar

1.0 1.5 13.5 33.0 50.9

The Ability to Entertain Multiple Conflicting Views

1.0 2.1 13.1 35.8 48.0

Knowledge of English and American Literary Traditions

8.3 14.0 26.8 26.7 24.1

Knowledge of Global Literary Traditions

12.3 21.5 29.5 21.7 15.0

Table 2. Relevance of Skills to Professional Development

J357-378-Mar14-CE.indd 369 1/20/14 3:54 PM

370 College English

derived instrumental, professional utility from reading and thinking about Shake-speare as college students. To put it a different way: Someone’s closing deals with Hamlet’s help.

Designing a quantitative survey of a fifty-year cohort of alumni that would provide us with useful information necessitated careful choices about the kinds of questions we could ask and limited the information we could cull from the results. However, we were able to add some depth to our findings by taking advantage of the fact that Internet-based surveys tend to elicit from respondents detailed answers to open-ended questions (“Telephone”). In order to better learn about the instru-mental value of the apparently noninstrumental knowledge we impart, we included in the survey a fairly extensive qualitative component in which alumni were asked questions that invited them to describe the “parts of your education [that] were most important in your career.”

We reviewed the collected responses and sorted them in terms of what we are calling habits of mind. Though we initially borrowed this term from educational researcher Arthur Costa, who coined it to describe what intelligent people do when they are confronted with problems whose solutions are not easily apparent (Costa and Kallick 395–401), it is useful to note that a collaborative task force from the Council of Writing Program Administrators, the National Council of Teachers of English, and the National Writing Project also uses this term in a similar if more expansive manner in the Framework for Success in Postsecondary Writing, defining habits of mind as being “ways of approaching learning that are both intellectual and practical” (1).

We wanted to use alumni’s own descriptions of what they have preserved and used from their English degree in order to determine what we pass on through our disciplinary heredity. That is, what are the mental habits or intellectual and practical strategies woven into the culture of our courses, the reading and writing we assign, and the responses and evaluations we give (Kekale 217–38)? Despite the seemingly infinite range of possible answers, a fairly discrete set of themes emerged to describe the ways in which being an English major is “used” by our alumni.

According to the language provided by the respondents, the habits of mind learned from studying English that are most important to their current careers are analytic thinking, writing persuasively, evaluating information, and seeing others’ points of view. We matched respondents’ answers with their professions because we were interested, for purposes of this study, in the less obvious professions for English majors: those who don’t go on to be high school teachers or college professors or technical and creative writers. That is, we wondered about the business owners, real estate brokers, therapists, mortgage lenders, and government agents. How might their responses shed some light on a path, however circuitous, from our courses in Mark Twain and Chaucer to our alumni’s subsequent professional development?

J357-378-Mar14-CE.indd 370 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 371

Rather than conduct statistical analyses as we did with the quantitative portion of the survey, our goal with the written commentary was to create a rough portrait of how our respondents answered the questions we posed. In the tables that follow, we include comments that we thought would best illustrate the kinds of responses we received but that would also demonstrate the many and unforeseeable ways in which the experience of studying English resonated through the lives of our alumni, regardless of the decade in which they graduated.

Most commonly cited as being important to one’s current career was learning to write and use correct grammar. This response was consistent across career paths and graduation cohort. In addition to this general claim about the value of learning to write, many respondents noted the process of critical analysis, which was cited numerous times for its value to an array of careers. (See Table 3.)

A second theme that emerged from the qualitative response was the impor-tance that the process of evaluating information has to alumni’s current careers. (See Table 4.)

A third kind of thinking that our graduates claim to have learned in the English classroom and use in their current profession is the ability to learn and see from others’ perspectives. (See Table 5.)

One final habit of mind that our former students identified as useful in their careers is perhaps the most surprising: knowledge about literature. (See Table 6.) Some components of an English degree—knowledge of grammar, rhetoric, and writing—seem to have clear instrumental value. There are few fields, after all, in which communication is not valued in some capacity. But there are comparatively few professions in which knowledge of literature would likely have instrumental value. And yet certain respondents felt that it was studying literature that was “most important” in their careers—even when those careers seemed to have nothing at all to do with literature.

On the surface, what we learned from the qualitative section of the survey simply reinforced the quantitative section. The qualities of an English degree that were rated professionally valuable by respondents in the closed-ended questions paralleled the qualities described in the open-ended questions. But what the qualitative data added were alumni evaluations about where and how they used the habits and skills they learned from us. The utility of such habits and skills emerged in ways neither we nor they would likely have anticipated. If we could turn back the clock and ask the young man who would become a fraud investigator how he would be using his English degree, we suspect that he would not have told us he aspired to be a fraud investigator and, more important, that he had no idea that there was a connection between reading Faulkner’s The Sound and the Fury and conducting civil and criminal investigations. The graduating senior who would become an eligibility technician can only now look back to trace the manner in which “what constitutes genuine research and valid

J357-378-Mar14-CE.indd 371 1/20/14 3:54 PM

372 College English

Title or Profession (Graduation Year)“What is your current job title?”

Applied Learning“What parts of your education were most important in your career?”

Purchasing Manager (1960)

“Thinking and analyzing.”

Real Estate Broker (1971)

“Ability to read, interpret, analyze, and communicate.”

Director of Prospect Research (1972)

“Analytic thinking, considering alternative points of view/ opinions.”

Consul (US Foreign Service) (1976)

“Training in analytic and communication skills were critical to my last career as a consular officer [. . .]”

Business Underwriting Specialist (1977)

“The emphasis on analytical reading and writing. Learning to view the world creatively and objectively.”

Executive Legislative Representative (1980)

“Reading and analysis—section by section—the meaning of work of the literary masters.”

Biofeedback Therapist(1986)

“The basic thinking skills, logic, interpretation, and evaluation. Each piece of the puzzle contributes to the whole picture. Every really worthwhile idea of any culture is captured in its literature and art. Being able to interpret ideas helps one to discover one’s self.”

Management Analyst (1991)

“My ability to write, synthesize information, and gather re-search.”

Key Account Representative(1991)

“The ability to analyze data, entertain different points of view and to use that information to better serve my customers, my company and myself.”

Fraud Investigator(1996)

“The acquisition of critical thinking, research, and language skills.”

Structure Planner(2003)

“If I had to choose, it would be the ability to become more methodical and be able to succinctly and concisely communicate a particular point of view.”

Support Services(2005)

“The skills learned in critical and analytical thinking, the arena of cultural self-critique, and the improvement of writing skills have been extremely valuable.”

Medical Sales(2008)

“Understanding literary works and dissecting them from start to finish. I do this daily with people and sales, instead of literary works. It also helped tremendously to understand the medical books, terminology, and disease states that I had to learn for my current position when I had no medical or science background.”

Table 3. Applied Value of Critical Analysis to Subsequent Career

J357-378-Mar14-CE.indd 372 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 373

Table 4. Applied Value of Information Evaluation to Subsequent Career

Title or Profession(Graduation Year)“What is your current job title?”

Applied Learning“What parts of your education were most important in your career?”

Eligibility Technician(1979)

“[T]he most important aspect of my education was the recogni-tion of what constitutes genuine research and valid information. While this may sound somewhat abstract, I believe the process of critical thinking, and researching and validating that thinking, remains the most important valuable aspect of my education, regardless of what field I have worked in.”

Business Owner(1987)

“It is extremely important to know how to write, and to be able to evaluate a written piece, and discern if the logic behind it holds true.”

Financial Aid Counselor(2005)

“Analysis of information/data/statements and state regulation, particularly as it pertains to ‘professional judgment.’”

Table 5. Applied Value of Ability to See Others’ Perspectives

Title or Profession(Graduation Year)“What is your current job title?”

Applied Learning“What parts of your education were most important in your career?”

Pharmacy Manager(1970)

“The ability to work with people and analyze a problem and write it up.”

Treasury Representative(1970)

“Diversity of outlooks.”

Director of Prospect Research(1972)

“Analytic thinking, considering alternative points of view/opin-ions.”

Key Account Representative(1991)

“The ability to analyze data, entertain different points of view and to use that information to better serve my customers, my company and myself.”

Ticket Sales(1999)

“Accepting different view of opinions and learning about other cultures from a literary stand point were the most important to me, as well as the challenges of research papers. This education helped me to be more detailed in my job position and accept other points of view.”

Associate VP of Scholarship Technology(2004)

“The challenge of exploring new subject areas, developing as a writer/researcher, presenting work in novel and creative ways, critical group discussion.”

J357-378-Mar14-CE.indd 373 1/20/14 3:54 PM

374 College English

information” would become critically important to a career position that “discusses information with individuals such as payment methods, eligibility requirements, pos-sible payments and applicants’ legal rights to dispute claims” (“Job Description”). The applied value of college learning, in other words, is not static, but is constantly evolving over students’ entire lives.

F r o m N o N i N s T r u m e N T A L K N o w L e d g e T o i N s T r u m e N T A L s K i L L s

Like many educators and educational researchers, Arum and Roksa are concerned about students and faculties being “adrift.” A tool historically used to prevent such drifting was the sextant, a navigational instrument that calculates the angle created by the horizon and the sun or another celestial body in order to determine the ob-server’s position line on a nautical chart. One way of reconceptualizing assessment is to think of students as ships orienting themselves by their university degrees and the horizon of their careers. If we are to truly understand the value-added of an undergraduate education, we need to determine if former students can locate them-selves in relation to their college educations, to ask them to chart a position line

Table 6. Applied Value of Literary Knowledge

Title or Profession(Graduation Year)“What is your current job title?”

Applied Learning“What parts of your education were most important in your career?”

Managing Broker(1972)

“Learning about the work of specific authors.”

Facilitated Enroller(1982)

“The English program expanded my knowledge of language and literature and made me a better reader, writer, poet, and person.”

Associate Government Program Analyst(1988)

“Just the study of English in both the literary and practical usage forms helps with understanding people better and communicating with them through verbal and written applications. Because of this, the English program at CSUF helped me in both my professional and personal lives.”

Fraud Investigator(1996)

“Oddly enough, I’ve found that explicating a poem or understand-ing, say, The Sound and the Fury, was the perfect training ground for civil or criminal investigation.”

Computer Systems Analyst(2002)

“The passion for literature and knowledge.”

Project Coordinator(2008)

“Contemporary Novels in English, Intermediate Creative Writing, American Literature.”

J357-378-Mar14-CE.indd 374 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 375

triangulated from the horizon of their professions and the educations they received ten, twenty, or fifty years ago.

One implication of our findings is that there is a sort of transmutation from noninstrumental to instrumental knowledge that happens outside the classroom. When professors teach Faulkner’s The Sound and the Fury, it is as a noninstrumental object of study. It is learned and appreciated for its own sake, or for its place in a larger academic conversation. It is not taught for its instrumental qualities: attributes for which a student might put it to subsequent professional use. Nor, for that mat-ter, could it be taught in such a way. (Imagine the infinitude of the course catalog: “Faulkner for Fraud Investigators”; “Faulkner for Heart Surgeons”; “Faulkner for HVAC Repairpersons.”) The results of our survey suggest that students eventually come to find instrumental value in noninstrumental learning. This is a dimension of student learning that is entirely overlooked by the current administrative conversa-tions about student learning assessment.

Student assessment in the modern university walks a tricky line between in-strumentalism and noninstrumentalism. Even though assessment is instrumental in its assumptions—operating on the premise that all learning can be applied and then measured—program assessment nonetheless is frequently tasked with measuring noninstrumental knowledge, which is the core of liberal higher education. This is an important point, yet one that is surprisingly easy to forget. Even disciplines that might superficially appear to focus on instrumental knowledge, such as nursing or accounting, are in fact in the business of the academic study of that instrumental knowledge. That is to say, they impart noninstrumental knowledge about instrumen-tal knowledge. Any academic discipline, properly understood, is structured by such noninstrumentalism. HVAC repair is vocational instrumentalism, but the theory of HVAC repair is noninstrumental learning. As the old joke goes, garbage is garbage, but the history of garbage is the stuff of academic study.

On the one hand, the noninstrumental makeup of academic knowledge means that student assessment must also focus on such noninstrumental learning. But on the other hand, comprehensive student assessment is nearly always (and perhaps necessar-ily) explained by its instrumental implications. For many critics of higher education, college learning is intertwined with technological and social progress generally, and global economic competitiveness specifically. We have to measure academic learning, goes this reasoning, because academic learning is not merely academic learning. At some point, it becomes applied. But if this is true, then assessment measures must take into account the applied or instrumental value of academic or noninstrumental learning. If they do not, then those measures are negating their own assumptions.

It is common to think of forms of direct assessment as being the most accu-rate: the kind of assessment that most exactly measures whatever it is we set out to measure. If we believe that the value added by a college degree is best measured by

J357-378-Mar14-CE.indd 375 1/20/14 3:54 PM

376 College English

assessing the extent to which students perform sets of skills within the harbor of our campus, then assessment measures like the CLA are arguably useful and sufficient. But if we want to know with any confidence whether that education adds lasting value to the lives of students, then the value of indirect, retrospective assessment cannot be overlooked. The value-added of a college degree may begin in students’ sophomore years, but it does not end until many years later. More important, the quiddity of that value can only really be assessed by the students themselves. To have a more complete understanding of the long-term value of a college education and the multifold ways in which students go on to transform noninstrumental study into instrumental value, we should simply ask them. If student assessment is justified on instrumental grounds—that student learning serves our economy or democracy—then assessment cannot concern itself solely with noninstrumental matters. Students must tell us which of a thousand celestial bodies we can align to see the path they took from our harbor to their own Portofino.

N o T e s

1. We thank the staff of the CSUF Social Science Research Center for their help in creating, dis-tributing, and analyzing the survey; the Dean’s Office of the College of Humanities and Social Sciences and the Office of Assessment and Educational Effectiveness for their funding; and our graduate student assistant, Patrick Vallee, and the English department administrative assistant, Maria Figueroa, for their contributions to our research.

w o r K s C i T e d

Allen, Mary J. Assessing Academic Programs in Higher Education. Bolton: Anker, 2004. Print.Arum, Richard, and Josipa Roksa. Academically Adrift: Limited Learning on College Campuses. Chicago: U of

Chicago P, 2011. Print.Attewall, Paul. “Riddle Remains Academically Adrift.” Society 48.3 (2011): 225-226. Web. 4 Feb. 2013.Auletta, Ken. “Get Rich U.” New Yorker 30 Apr. 2012. New Yorker. Web. 23 Jan. 2013. < http://www.new

yorker.com/reporting/2012/04/30/120430fa_fact_auletta>.Bauerlein, Mark. “Charting Academic Drift.” Academic Questions 24.3 (2011): 352–58. Academic Search Premier.

Web. 23 Jan. 2013.Bérubé, Michael. “Don’t Know Much about a Science Book, Don’t Know Much about the French I Took

. . .” Review of Politics 74.2 (2012): 364–66. Academic Search Premier. Web. 4 Feb. 2013. California State University Fullerton Integrated Strategic Plan. California State University, Fullerton, n.d. <http://

planning.fullerton.edu/planning/_resources/pdf/CSUF-Strategic-Plan.pdf>. Web. 20 Dec. 2011. Carey, Kevin. “‘Trust Us’ Won’t Cut It Anymore.” Chronicle of Higher Education 18 Jan. 2011. Chronicle of

Higher Education. Web. 23 Jan. 2013. Caspers, Jean. “What Happened to Academic Rigor?” Miller-McCune 4.2 (2011): 47-49. Communication &

Mass Media Complete. Web. 1 Mar. 2013.Chesbrough, Ron. “The Sky Is Falling, Or Is It? Remaking the Case for Student Affairs in Higher Education.”

Student Affairs Leader 39.8 (2011): 2-8. Academic Search Premier. Web. 13 Jan. 2013.Costa, Arthur L., and Bena Kallick. Learning and Leading with Habits of Mind. Alexandria: Association for

Supervision and Curriculum Development, 2000. Print.

J357-378-Mar14-CE.indd 376 1/20/14 3:54 PM

Closing Deals with Hamlet’s Help 377

Council of Writing Program Administrators, National Council of Teachers of English, and National Writ-ing Project. Framework for Success in Postsecondary Writing. N.p.: CWPA, NCTE, and NWP, 2011. <http://wpacouncil.org/files/framework-for-success-postsecondary-writing.pdf>. Web. 1 Mar. 2013.

CSUF 2012 Fact Sheet.” California State University, Fullerton, <http://calstate.fullerton.edu/multimedia/factsheet.pdf>. Web. 25 Oct. 2013.

Fontaine, Sheryl I., and Stephen J. Mexal. “The Starbucks Myth: Measuring the Work of the English Ma-jor.” ADE Bulletin 152 (2012): 36–46. <http://www.ade.org/bulletin/index.html>. Web. Apr. 23. 2013.

Glenn, David. “Scholar Raises Doubts about the Value of a Test of Student Learning.” Chronicle of Higher Education. 2 June 2010. Chronicle of Higher Education. Web. 13 Jan. 2013.

Goral, Tim. “Academically Adrift: Limited Learning on College Campuses: A Review Essay.” University Business 14.2 (2011): 20. Academic Search Premier. Web. 23 Mar. 2013.

Hacker, Andrew, and Claudia Dreifus. Higher Education? How Colleges Are Wasting Our Money and Failing Our Kids—And What We Can Do about It. New York: St. Martin’s Griffin, 2011. Print.

Hogan, Eileen A., Anna Lusher, and Sunita Mondal. “Development of an Instrument for Indirect Assessment of College Business Programs.” Journal of Case Studies in Accreditation and Assessment 2, Pt. 1 (2012): 1–15. Print.

Horowitz, Damon. Manage Your Career. “From Technologist to Philosopher.” Chronicle of Higher Education 17 July 2011. Chronicle of Higher Education. Web. 23 Jan. 2013.

Hosch, Braden J. “Time on Test, Student Motivation, and Performance on the Collegiate Learning Assess-ment: Implications for Institutional Accountability.” Paper presented at the Association for Institu-tional Research Annual Forum, Chicago. June 2010. < http://www.ccsu.edu/uploaded/departments/AdministrativeDepartments/Institutional_Research_and_Assessment/Research/20100601a.pdf >. Web. 1 Mar. 2013.

Igo, Sarah E. “Academics Adrift?” Academe 97.5 (2011): 53-56. Academic Search Premier. Web. 23 Jan. 2013.“Job Description of an Eligibility Technician.” eHow.com. Demand Media, 25 June 2010. <http://www.ehow

.com/about_6652915_job-description-eligibility-technician.html >.Web. 23 Jan. 2013. Kekale, Jouni. “‘Preferred’ Patterns of Academic Leadership in Different (sub)cultures.” Higher Education

37.3 (1999): 217–38. Academic Search Premier. Web. 1 March 2013.Lee, Stephen W., ed. Encyclopedia of School Psychology. Thousand Oaks: Sage, 2005. Print.Maki, Peggy. Assessing for Learning; Building a Sustainable Commitment across the Institution. Sterling: American

Association for Higher Education, 2004. Print.Menand, Louis. “Live and Learn: Why We Have Colleges.” New Yorker 6 June 2011: 74–79. Print. Menhart, Steve. “Direct Versus Indirect Assessment Methodologies.” Proceedings of the 2011 Midwest Section

Conference of the American Society for Engineering Education, Russellville, AK, Sept. 2011. <http://www.asee .org/documents/sections/midwest/2011/ASEE-MIDWEST_0004_315750.pdf>. Web. 1 March 2013.

Possin, Kevin. “A Fatal Flaw in the Collegiate Learning Assessment Test.” Assessment Update: Progress, Trends, and Practices in Higher Education. 21 Feb. 2013. < http://www.assessmentupdate.com/sample-articles/ a-fatal-flaw-in-the-collegiate-learning-assessment-test.aspx.>. Web. 1 Mar. 2013.

“Reviews of Academically Adrift: Limited Learning on College Campuses.” CCC 63.3 (2012): 487–90. National Council of Teachers of English. Web. 10 Jan 2013.

Rudolph, Frederick. The American College and University: A History. New York: Alfred. A. Knopf, 1962. Print. Sternberg, Robert J. “Who is Really Adrift?” Inside Higher Ed. 8 Feb. 2011. Inside Higher Ed. Web. 10 Jan 2013.Suskie, Linda. Assessing Student Learning: A Common Sense Guide. San Francisco: Anker, 2004. Print.Swail, Watson Scott. “Measuring Learning.” Swail Letter on Higher Education. 18 Nov. 2011. Educational

Policy Institute. Web. 12 Jan. 2013.“Telephone Surveys vs. Web Surveys: A Comparison.” Telephone Surveying Vs. Web Surveying. Ipathia Inc.,

n.d. < http://knowledge-base.supersurvey.com/phone-vs-web-surveys.htm>. Web. 23 Jan. 2013.

J357-378-Mar14-CE.indd 377 1/20/14 3:54 PM

378 College English

Thelin, John R. A History of American Higher Education. 2nd ed. Baltimore: Johns Hopkins UP, 2011. Print.Yamayee, Zia A., and Albright, Robert J. “Direct and Indirect Assessment Methods: Key Ingredients for

Continuous Quality Improvement and ABET Accreditation.” International Journal of Engineering Education 24.5 (2008): 877–83. Print.

J357-378-Mar14-CE.indd 378 1/20/14 3:54 PM