Last spring, we began a review of our work to date with Wabash National Study institutions. The primary goal of the Wabash National Study is to help institutions gather and use evidence to improve their impact on student learning. We collaborate with our colleagues at Wabash Study schools like Coe College, North Carolina A&T, and Lasell College on issues ranging from ways of interpreting the evidence and how best to get the evidence into the hands of staff and faculty, to considering what changes would have the most benefit or are most plausible given the available resources. In working through our review, we considered both the obstacles and breakthroughs that we've encountered in our work together.

The biggest surprise for us was how much high-quality assessment data most campuses already have. As Charlie said in response to a reporter's question, "In every case, after collecting loads of information, we have yet to find a single thing that institutions didn’t already know." (see Inside Higher Ed article) This finding certainly came as a surprise to people who've spent a good part of their recent working lives trying to implement a complicated study that hands institutions mountains of new assessment data, and it has led us to ask questions about whether there are downsides to "faculty friendly" assessment or "good campus conversations" about student learning. We explore the potential pitfalls of these common approaches to assessment, and discuss practices that can help institutions move beyond the cycle of gathering and having conversations about data to using data to improve student learning, in a recent report on the progress of two assessment collaboratives. If you are interested, you can read the entire report (PDF).

If time is short and you'd like to get to the heart of the matter, you can see the list of good practices that, in our view, help institutions move from gathering assessment data to actually using it to help student learning below:

• Plan for how you will respond to the evidence before you get it
  
  • Have a "syllabus" outlining the activities you plan for the upcoming year. The syllabus should outline:
    • The different ways you plan to disseminate the information
    • How you plan to move the assessment information through campus governance structures
    • Resources that you've allocated to respond to good ideas that emerge in response to the evidence

• Disseminate the evidence in multiple ways and in different places
  • Don't pop it on a website and hope that someone will read it. Develop a communication plan for how you will move the information through your campus and designate who will be responsible for moving it.

• Enlist strong administrative support
  • Strong, supportive, and consistent campus leadership is essential for using evidence to make changes that improve student learning. What can administrators do to help?
    • Protect assessment leaders from colleagues who are morally opposed to assessment.
    • Provide resources to assessment leaders, faculty, and staff who are gathering, considering, and using evidence on student learning.
    • Help guide assessment evidence through institutional governance structures.
    • Publicly articulate their support for assessment and campus assessment leaders.
      This public articulation is most effective when it takes an improvement-oriented perspective and focuses on using assessment to strengthen the institution, rather than the defensive stance of using it to keep accreditors and other outsiders at bay.

• Collaborate with students to help interpret assessment evidence
  • Incorporating students into the process of making sense of assessment evidence will increase the quality of your interpretations of assessment data and it will add student voice and perception to that data in a way that will connect with staff and faculty. You can do this in a variety of ways:
    • Ask groups of students to look at and interpret assessment evidence.
    • Raise the questions or concerns you have about assessment evidence with students and ask for their perspective.
    • Create faculty, staff, or student-led focus groups to ask students about experiences that have been helpful to them and experiences that have held them back. Ask students what they think the mission of the institution is or to name the one thing it teaches best.

• Include an evaluation of student academic work in any assessment program, regardless of how many national tests or surveys you are using
  • Not only do student papers, projects, presentations, performances, and other kinds of academic work provide useful information about student learning, examining student work connects with faculty and staff far more than any standardized test.

• Use multiple methods to understand evidence about student learning
  • Each assessment methodology has strengths and weaknesses. Using different methods to probe assessment questions not only improves the quality of evidence but invites faculty and staff from disciplinary traditions to engage the evidence.
  • However, avoid "methodological puritanism" or "method idolatry." In many cases, assessment evidence can help you make changes that will improve student learning even if the evidence does not meet the highest scholarly standards. The goal of assessment is to improve the institutions, not fatten our vitae with more publications. Don't let the perfect be the enemy of the good enough.

• Think twice before you gather more data
  • At most institutions, the real problem is not the lack of good assessment evidence, it's the lack of action.
    • In every case the interesting and surprising facts brought to light by the Wabash National Study were foretold by evidence that campuses already had. There should have been no surprises.
  • Do you know what you already have? Have you thought about what it tells you?
    • Useful assessment evidence comes not only in the form of national survey data or sophisticated outcomes measures, but also from institutional grades, graduation rates, enrollment patterns, student exit interviews, interviews with internship hosts, and so on. We can use institutional records and student work to get precise information about how our students are doing. This will help us move beyond the uninformed hunches or unquestioned myths that often drive our decisions. For example, do you know with precision which courses on your campus have high DWF rates? Do you know with precision the level of academic success of different groups of students on campus? Do you know with precision what senior capstone projects or final projects have told you about how your students are learning?
    • One campus we're working with initially thought they had no useful survey information. After a search, it turned out they had recently participated in over a dozen different national surveys, many of them completed multiple times. To our knowledge, these high-quality and expensive surveys had never been used for assessment.
  • Be wary of "we need more data before we can act."
    • Sometimes it is true that gathering more data will help us decide what action to take, but often gathering more data is simply a safe way of forestalling action.
    • There's an alternative to gathering more data – Make a change and see what happens.

 Click here for printer-friendly version.

--CB & KW

In late October, the National Institute for Learning Outcomes Assessment will release the results of a national survey on why, how, and to what level colleges and universities assess student learning. A significant finding from this survey is that institutions cite accreditation is the primary reason they assess student learning. Not a surprising result.

A key accreditation requirement is that departments have to develop and implement assessment plans. In this issue of Assessment Notes, Vicki Baker (Albion College) and Marie Kendall Brown (University of Louisville) describe how Albion’s Department of Economics and Management developed an interesting and useful assessment plan. They also make a number of helpful recommendations for faculty who are building departmental assessment plans.

In addition to a clear description of the process of developing an assessment plan and good suggestions for others engaged in this process, two things about Baker and Brown’s work caught our attention. First, we were impressed by the extent to which Albion’s Economics and Management Department enlisted the help of people who employ Albion graduates in developing departmental learning outcomes. This is a good practice that departments can use, whether they aim to help students move on to post-graduate education, post-graduate employment, or both. Getting information about how “outsiders” view the strengths and weaknesses of your students, as well as the requirements of the different worlds your students enter after they graduate, can be enlightening. 

Second, we enjoyed Baker and Brown’s argument that “reframing the liberal arts and vocational debate to acknowledge the similar goals that both traditions espouse invites productive conversation.” Many liberal arts advocates understand liberal arts education as focusing on the content of the disciplines of the arts and sciences. But assessment invites us to take a different perspective because it asks us to focus on the results, not the means, of education. If we learn through assessment that students who major in business or nursing are just as likely to become “graduates who will be responsible and ethical citizens who demonstrate leadership skills, utilize critical thinking, and who have a lifelong inclination to inquire” as are students who major in the arts and sciences, why aren’t these applied majors just as much a part of liberal arts education as traditional majors in the arts and sciences? What if, at some institutions, applied or vocational majors are more effective than arts and sciences majors at helping to develop the kinds of qualities of mind and heart that characterize a liberally educated person? Perhaps one of the reasons that assessment is so controversial is because it can lead us to reframe questions in a way that subverts our comfortable ways of thinking about education.

If you are interested in reading more about the relationship between professional and liberal education, you might want to look at:

Bill Spellman - The Resilient Liberal Arts College

Lee S. Shulman - Professing the Liberal Arts

Robyne Hart - The Tortoise, the Hare, and a Better Approach to Business Education

Steve Weisler et al - Liberal Education and Preparation for a Life of Work, a Teagle Foundation White Paper (PDF)