Assessment Notes

 Center of Inquiry Home

September 23, 2009

Possible Connections between Second Language Acquisition and High Performance on the Collegiate Learning Assessment at Kalamazoo College

by Katie MacLean, Jennifer Redmann, Kathleen Smith, and Jan Solberg

Like other institutions of higher education, Kalamazoo College endeavors to provide the best education possible for its students. In so doing, carrying out periodic formative assessment of student learning is paramount, and applying what is learned through assessment, though more difficult to accomplish, helps to improve the education provided. As part of our assessment efforts, Kalamazoo College administered the Collegiate Learning Assessment (CLA) to first-years and seniors in 2005 and 2006, and again to seniors in 2007. By disaggregating student CLA scores and then grouping them by academic division, we revealed that foreign language majors at Kalamazoo College scored significantly higher on the CLA (Sotherland, 2009, see graph). While considering possible causes for this phenomenon, we reviewed recent literature on ways in which second language learning might enhance competency in the native language as well as critical thinking and problem-solving skills. The review revealed four attributes—metalinguistic awareness, critical reading, critical thinking and problem solving, and cross-cultural literacy—that receive explicit and regular attention in language courses and that may contribute to better performance on tasks like the CLA. We will describe these four attributes, share insights gained from trying to understand the reasons for disparity among student scores on the CLA, and suggest ways we can use what we learned to improve student learning.

What is the Collegiate Learning Assessment and what can it tell us about our students’ education? The CLA performance assessments are “authentic problems or simulations of real world issues” that are administered online and that require examinees to use “an assortment of documents such as tables, figures, graphs, newspaper reports and photographs” (Benjamin et al., 2009, p. 3) to compose and support their answers. Scoring of students’ responses “provides rich diagnostic information about the students’ writing, analysis, and critical thinking skills” (Benjamin et al., p.3). Results from the CLA, especially when correlated with other measures of what occurs during a college education (e.g., NSSE Deep Learning Experiences, Appendix 1) can be helpful when deciding how to modify facets of a college education to enhance learning.

Metalinguistic awareness

According to the American Psychological Association, metalinguistic awareness “is a conscious awareness of the formal properties of language as well as its functional and semantic properties. It is associated with a mature stage of language and metacognitive development . . . and is often signaled by an interest in puns and word games.” (American Psychological Association, 2007) An example would be the realization that, to use the terms of Saussurian linguistics, the relationship between the signifier and the signified (between the sound/image and the concept) is arbitrary (Saussure, 1986). Nothing about T-R-E-E is inherent in the image we conjure in our minds when we hear or read the word tree. Many studies of bilingual children and children who are learning a second language have shown that exposure to and competency in more than one language increase metalinguistic awareness, partly because knowledge of two language systems underscores the arbitrariness of the above-mentioned relationship. Several studies (Alanis, 2000; Lazaurk, 2007; Ransdell et al., 2006; Ricciardelli et al., 1989; Smith, 1981; Yelland et al., 1993) have tracked measurable and valuable benefits to both reading (word recognition, comprehension of complex sentence structures) and writing (vocabulary range, sentence complexity).

Classroom language learning, which focuses on vocabulary development and sentence structure, appears to result in improved writing ability in the native language because of increased metalinguistic awareness. At Kalamazoo College, all students are required to reach the low intermediate level in a foreign language because we value cross-cultural communication and awareness of international viewpoints, and because over eighty percent of our students study abroad and must apply these skills while living in another culture. However, the performance of senior foreign language majors on the CLA suggests that a higher level of competency in another language provides benefits that have little to do with the particular language and culture studied. Sanders (2006–7) recently reminded us that there is abundant evidence to show that “a foreign language learned in a classroom improves the learner’s native language competence as well” (p. 41). When a student realizes that there is no word-for-word equivalency between English and the second language, he or she is forced to consider and compare deeper structures of both languages. Sanders refers to the research of Kecskes and Papp (2000) who demonstrate that “people with more than one language have different knowledge of the first language than do monolingual people, and that this difference can mainly be due to the effect of subsequent languages on the development and use of [native language] skills” (Kecskes & Papp, 2000, as cited in Sanders, 2006–7, pp. 41–42). Therefore, learning a second language may, through increased metalinguistic awareness, improve performance on CLA tasks.

Critical reading

Another hypothesis that might help account for the performance of foreign language majors on the CLA is that these students are resensitized to the reading process in their language courses in a way that makes them very aware of the structure, rhetorical trajectory, and word choice of any text they read. Because even the most advanced students may struggle with sophisticated texts in a foreign language, instructors revisit the reading process in slow motion, demonstrating various strategies to help them decode an unfamiliar text.

For example, a typical approach to reading a text in an intermediate-level language class includes most or all of the following:

  1. An exercise designed to activate prior knowledge of the subject about which they will read (e.g., “What 20th century events contributed to changes in the status of Western women?” or “You are a physician who must help a family decide whether or not to keep a patient on life support. What factors will you bring up with the family?”).
  2. Vocabulary exercises focusing on words in the text that students might not know, but that are essential to comprehension. These exercises might include multiple-choice or fill-in-the-blank items; giving definitions or using the words in student-constructed sentences; making lists of related words (e.g., tempo, temporary, contemporary, extemporaneous); thinking of all the ways in which a particular word might be used (e.g., turn right, human right, political right, to do right, etc.); understanding the nuances of a word used in another cultural context (e.g., for the French, the word collaboration still evokes support of the Vichy government during World War II). 
  3. Exercises that make students aware of grammatical nuances, complex sentence structures, or allusively expressed ideas (e.g., in French, “It would have been with the president’s approval” means “That’s one theory, but I don’t believe it” or “It’s never been confirmed.”; in English, double negatives can cause comprehension problems for non-native speakers [“He can’t not do it!”]; the passing observation in a text that all couples on a TV show are heterosexual is intended as a critique of the nonrecognition of homosexual couples, etc.)
  4. Exercises that ask students to predict content of the text by looking at the title and illustrations, and by thinking about where the text appeared and the year in which it was published, etc.
  5. A first cursory reading in which students respond to a few straightforward comprehension questions (the answers to which are verified before proceeding to closer readings).
  6. Another reading (or readings) in which students are asked to find a thesis statement (not always in the first paragraph in texts produced in other cultures), find the main idea of each paragraph, consider what sort of examples or evidence are cited to support the main idea (certain things are considered appropriate evidence in other cultures that we would not accept as such in mainstream U.S. American culture). Students might then be asked to consider the macrostructure of the text. Are conclusions stated in the first paragraph or later in the text? How are talking points presented (strongest to weakest? weakest to strongest? interspersed with counterexamples? Is there an attempt at synthesis?) What happens in the conclusion—does it restate the original thesis or do something else?
  7. A similarly analytical examination of the text will also be carried out on the microlevel, looking, for instance, at the types of rhetorical markers designed to guide the reader. Where do we see such words as for example, on the other hand, thus, and in conclusion? Are they used correctly? Are there words that have significant connotations (such as the word collaboration in French)? Are there words that are value-laden (primitive, dialect, bad, liberal, etc.), and does the author seem aware of judgments implied by these words?
  8. A thorough discussion of the content of the text that examines the writer’s ideas, evaluates the strength of the argument, asks students to express their own ideas on the subject, and pushes beyond the text to apply the writer’s ideas in another context.
  9. Students may be asked to produce their own version of the text. This could be in the form of a debate (one student expresses the ideas of the writer, and another takes an opposing point of view), a “letter to the editor” in which the student reacts to the original text, or the student’s own version of a similar text (same structure on another subject, same subject in another genre, etc.). This “production” process often involves a critique of the student’s initial draft and the opportunity to revise.

Of course, we expect all college students to be able to interact with a text in these ways, but we often take it for granted that they have learned—and will use—a methodical, critical approach to reading. Because of the difficulties inherent in reading texts in a foreign language, we explicitly teach or remind students about steps one can take to decode and reflect upon a text, and about the parallel processing involved in the act of reading. This might be likened to learning to drive a car with standard transmission after driving only cars equipped with automatic transmission. There is an initial clumsy stage of analytical awareness of the process that must be gotten through before it becomes ingrained and habitual once more.

We can be fairly confident that all foreign language students learn (or relearn) this process. This awareness of the reading process is so widespread in foreign language instruction that it permeates all textbooks currently used in the field. Once students begin to read more sophisticated texts, this pedagogical apparatus is not always supplied by publishers, but it is often provided by the instructor. Thus, making explicit best practices for reading may transform foreign language students into more competent readers in their native languages and help them perform better on the CLA.

Critical thinking and problem solving

Studying a foreign language requires linguistic problem solving, which, in turn, leads students to develop abstract and conceptual thinking: “Speakers . . . need to assess the critical aspects of the communication, marshal their abilities, and stretch the mastered elements of the language to achieve communication” (Sanders, 2006–7, p. 43). Working in a second language also requires students to be aware of the intentions and beliefs of other speakers in the target language with whom they will interact. This demands another dimension of critical thinking.

Awareness of context and of others in that context is central to student role-playing, a typical activity in a foreign language classroom. At beginning levels, this involves assuming a role in an already prepared dialogue and then creating a dialogue based on, or inspired by, the model. Here students begin to personalize models provided to them. At more advanced levels, students begin to adopt roles and points of view to enable them to understand various perspectives on values, attitudes, and beliefs. For purposes of conversation or debate, students adopt perspectives that may reflect their own values, or, on the contrary, put themselves in the place of persons with beliefs or points of view opposite to their own.

In the authentic setting that comes with study abroad, students engage in another form of role-playing: they are often called upon to imagine interactions with native speakers before engaging in the interaction. For example, imagine a student entering a small grocery store in France to purchase cheese. It is not prepackaged so the student will have to greet the grocer (as the French do), establish what he or she wants, be prepared to respond to possible questions (Is this slice too thick? Do you prefer this brand or that brand?). As the student prepares for this interaction, he or she mentally rehearses the range of possible questions and answers that the task will entail. In time, these rehearsals require less conscious effort and occur more naturally and spontaneously. The student eventually assumes a persona in the French interactive context that is different from her or his English-speaking persona. Because every interaction in an authentic environment is based on three elements—context, the role of other parties in the interaction, and the role the student will be called upon to play—the student becomes a different person in the setting.

Learning theory tells us that this kind of awareness of language in context enables higher levels of abstract thinking and conceptualizing (Sanders, 2006–7, p. 43). Students must be aware of language as a symbolic system and, simultaneously, must understand and manipulate the conceptual context. In a sense, the student works in a mode of divided attention when using a second language. When students become more adept at such divided thinking, they become more efficient in conceptualizing while communicating. These skills—verbal adeptness paired with imaginative deftness in context—are precisely the kinds of skills required in the problem-solving situations students encounter on the CLA.

Cross-Cultural Literacy

In most college-level foreign language classrooms, instructors provide students with an (often overwhelming) amount of language input: instructions, questions, dialogues, songs, written texts. Both inside and outside the classroom, students respond to this input by reading, answering questions, memorizing vocabulary, and practicing grammatical structures. However, the development of true language proficiency requires an additional step: the creative transformation of language input into new representations (Kern, 2000).

This creative language use begins in the earliest stages of instruction and continues to the highest levels of proficiency. Beginners spend a great deal of time learning to ask and answer questions about themselves, their families, and their interests. However, they quickly move beyond the personal world to situations in the target language culture. As students’ foreign language proficiency expands, the focus of classroom work increasingly shifts to engagement with texts, and all types of writing and speaking about those texts constitute forms of creative practice. For example, the creative transformation of texts in a second-year German course at Kalamazoo College reaches a high point at the end of the term when students present projects on one or more of the books they have read. Students create a board game, imagine a new ending for one of the stories, write a dialogue set years after the end of a book, put together a website or PowerPoint presentation on themes connected to the books, or even write and perform songs based on the plot of one of the books. This type of work requires a deep, multifaceted understanding of the text at hand, as well as the ability to draw on the language, structure, and style of the text to create something entirely new.

When texts form the foundation of student work, classrooms are transformed from places where students mimic the contexts of authentic language use to places where they speak about something real—the text before them—and together try to arrive at its various meanings. In this way, a group of students becomes a discourse community, different from the one that the author had in mind, but just as authentic. Kramsch and Nolden (1994) argue that the culturally determined gaps in students’ knowledge, gaps that interfere with their understanding of a foreign language text, provide a valuable opportunity for “oppositional practice” in the classroom.

Oppositional practice is not resistance, dissidence or contestation. It just claims the right of the readers to position themselves at equal par with, i.e., in (op)position to, the text, by virtue of the very linguistic and conceptual power that the text has given them. By becoming aware of their oppositional stance, readers can enter into dialogue with the text and with other readers and eventually, through this dialogue, experience “changes in desire” that potentially lead to social change. (Kramsch & Nolden, pp. 29–30)

When students enter into dialogue with the text and with each other, and use their position of outsider as a starting point for transforming the text, they begin to develop what Kramsch and Nolden describe as “cross-cultural literacy.” Kramsch and Nolden apply this term not to the exchange of products or ideas across cultures, but to the “relational process of border crossing itself” (p. 30). In this way, the very act of reading—the struggle to comprehend and respond to a foreign language text—allows students to travel into and participate in another culture while maintaining a sense of otherness and difference. This development of cross-cultural literacy may go hand in hand with the strong critical thinking skills that foreign language majors have demonstrated on the CLA.

Lessons Learned

In response to the CLA findings, members of the Division of Foreign Languages and Literatures at Kalamazoo are adopting a number of measures. In French, with the decision to no longer have a senior comprehensive exam, faculty are looking to create an instrument that will test language skills, understanding of cultural context, and perhaps problem solving. Results of the test will be used to improve the French program. The German department revamped its curriculum four years ago, offering a more holistic interweaving of language, textual analysis, and cultural understanding. Pursuant to external review of the Romance languages department, Spanish is fine-tuning its curriculum in this direction, while French is undertaking a major overhaul of its curriculum along these same lines, with greater emphasis on the cultural dimensions of the study of language and literature in order to better ensure the relationship between experiences of study abroad and the emphases of the on-campus French curriculum. Concern about insufficient development of writing skills while students are on study abroad has prompted discussion with the directors of the Center for International Programs to build a significant writing component into programs at the various sites. This will directly reflect the importance of skilled writing to the problem-solving component of performance on the CLA. This step is part of an overall effort to develop close reading skills of a variety of texts at every level of the curricula in French, German, and Spanish. Also, faculty are placing a greater emphasis on encouraging awareness of cultural context at every level, from beginning language to advanced literature and culture courses. At the beginning and intermediate levels, the use of role-play continues with its value confirmed in the kinds of tasks on which students excelled on the CLA. It should also be noted that faculty in the division have included in their course objectives not only the development of language and literacy skills but also more general analytical and research skills, writing and speaking in different registers, and making connections to other disciplines outside the course.

Examining a college education from different perspectives simultaneously, and iteratively, can lead to incremental improvements in student learning. We have done this at Kalamazoo with limited success in the past (Sotherland et al. 2007; Sotherland, 2009), but this summer we anticipate gaining clearer insights from longitudinal CLA results together with NSSE scores from the same students. Correlating differences in CLA performance, between students in their first year and again as seniors, with responses to questions about NSSE Deep Learning Experiences (which focus on integration, reflection, and some perspective taking; see Appendix) might provide additional insights into how educational experiences vary among students in different disciplines and how they effect change in abilities to analyze, reason, and write. If we find correlations that favor foreign language majors, we will examine transcripts of the students participating in the longitudinal study, looking for curricular features that might account for the observed correlations. We may also query students in the future using in-class surveys to see if pedagogical techniques or exercises (like CLA in the Classroom or structured reflection) are prompting similar learning experiences as those that seem to benefit foreign language students.

Pedagogical approaches used in the foreign language classroom (i.e., those that develop metalinguistic awareness, critical and creative interaction with texts, ability to adopt other perspectives, and cross-cultural literacy) could be adapted by other disciplines to enhance critical thinking, problem solving, and writing. For example, faculty in the natural sciences could include exercises that help students develop a conscious awareness of how they are learning, while faculty in the social sciences might remind students of strategies for reading texts more critically and deeply. Perhaps humanities faculty could use role-playing in a literature course to help students imagine how a character might react to a given piece of dialogue, or require students in the philosophy classroom to adopt opposing points of view. And, frequently encouraging students to carry out structured reflection on their learning, regardless of the setting, will help deepen the learning that occurs. Applying lessons learned from examining possible reasons for the high performance of foreign language majors on the CLA could provide richer educational experiences for more students and thereby enhance their performance on tasks like those encountered on the CLA.

Acknowledgements

We are grateful to The Teagle Foundation for two grants to Kalamazoo College that have fostered an environment where the scholarship of teaching and learning can flourish. And we thank Charlie Blaich, and other reviewers at the Center of Inquiry in the Liberal Arts, for useful feedback on this manuscript.

Literature Cited

Alanís, I. (2000). A Texas two-way bilingual program: Its effects on linguistic and academic achievement. Bilingual Research Journal, 24, 225–248.

American Psychological Association. (2007). APA Dictionary of Psychology. Metalinguistic awareness.

Benjamin, R., Chun, M., Hardison, C., Hong E., Jackson, C., Kugelmass, H., et al. (2009). Returning to learning in an age of assessment: Introducing the rationale of the Collegiate Learning Assessment.  http://www.collegiatelearningassessment.org/files/ReturningToLearning.pdf

Kern, R. (2000). Literacy and Language Teaching. Oxford: Oxford University Press.

Kecskes, I. & Papp, T. (2000). Foreign Languages and the Mother Tongue. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.

Kramsch, C. & Nolden, T. (1994). Redefining literacy in a foreign language. Die Unterrichtspraxis/Teaching German, 27, 28–35.

Lazaruk, W. (2007). Linguistic, academic, and cognitive benefits of French immersion. The Canadian Modern Language Review/La Revue canadienne des langues vivantes, 63, 605–628.

Ransdell, S., Barbier, M.-L., & Niit, T. (2006). Metacognition about language skill and working memory among monolingual and bilingual college students: When does multilingualism matter? The International Journal of Bilingual Education and Bilingualism, 9, 728–741.

Ricciardelli, L. A., Rump, E. E., & Proske, I. (1989). Metalinguistic awareness as a unitary construct, and its relation to general intellectual development, Rassegna Italiana di Linguistica Applicata, 21(1-2), 19–40.

Sanders, R. H. (2006–2007). Focus on form: Foreign language study and cognitive development. ADFL Bulletin, 38(1-2), 40–44.

Saussure, F. (1986). Course in general linguistics: Nature of the linguistic sign. In H. Adams and L. Searle (Eds.), Critical Theory Since 1965, (647–648). Tallahassee: University Press of Florida.

Smith, M.S. (1981). Consciousness-raising and the second language learner. Applied Linguistics, 2, 159–168.

Sotherland, P. (2009, September 23). Cogitating aggravating disaggregating. Assessment Notes.

Sotherland, P., Dueweke, A., Cunningham, K., & Grossman, R. (2007). Multiple drafts of a college’s narrative. Peer Review, 9(2), 20–23.

Yelland, G. W., Pollard, J., & Mercuri, A. (1993). The metalinguistic benefits of limited contact with a second language. Applied Psycholinguistics, 14, 423–444.

 

Katie MacLean, Kathleen Smith, and Jan Solberg are in the Kalamazoo College Department of Romance Languages, and Jennifer Redmann is in the Kalamazoo College Department of German

Assessment Notes is a publication of the Center of Inquiry in the Liberal Arts at Wabash College.

 


 

Appendix: NSSE Deep Learning Experiences

Higher-order learning

  • During the current school year, how much has your coursework emphasized: 
    • analyzing the basic elements of an idea, experience, or theory, such as examining a particular case or situation in depth and considering its components?
    • synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships?
    • making judgments about the value of information, arguments, or methods, such as examining how others gathered and interpreted data and assessing the soundness of their conclusions?
    • applying theories or concepts to practical problems or in new situations?

Integrative learning

  • During the current school year, about how often have you:  
    • worked on a paper or project that required integrating ideas or information from various sources?
    • included diverse perspectives (different races, religions, genders, political beliefs, etc.) in class discussions or writing assignments?
    • put together ideas or concepts from different courses when completing assignments or during class discussions?
    • discussed ideas from your readings or classes with faculty outside of class?
    • discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.)?

Reflective learning

  • During the current school year, about how often have you:  
    • examined the strengths and weaknesses of your own views on a topic or issue?
    • tried to better understand someone else's views by imagining how an issue looks from his or her perspective? 
    • learned something that changed the way you understand an issue or concept?