From the issue dated November 17, 2000
Survey offers way to assess undergraduates' satisfaction -- and possibly to evaluate colleges
By LEO REISBERG< P> While guiding prospective students and their parents on a tour of Elon College last month, Michael Garcia was greeted by a familiar faculty member.
"Hi, Mike," the woman said, "how's your knee doing?"
"Oh, is that your professor?" a parent asked Mr. Garcia, a senior majoring
It was certainly a convenient encounter, but one that Mr. Garcia says is typical at Elon, a comprehensive North Carolina college of fewer than 4,000 undergraduates.
The size of the campus facilitates close interactions among students and faculty members, he says. Professors encourage students to call them at home to discuss class assignments, grades, and life in general. In the classroom, active participation is more common than droning lectures.
If all this sounds like rhetoric one might expect to hear from a loyal student leader, a highly anticipated national study suggests he is not just blowing smoke. What some are calling a potential alternative to the college rankings, the National Survey of Student Engagement purports to be a new approach to measuring institutional quality.
Elon and three liberal-arts colleges -- Beloit, Centre, and Sweet Briar -- stood out among 276 institutions whose freshmen and seniors participated. The survey, a $3.3-million project sponsored by the Pew
|
Exemplary Institutions Following is an explanation offered by the researchers who prepared the National Survey of Student Engagement on identifying potentially exemplary institutions: There are different ways to think and talk about educational effectiveness. One way is to determine those institutions at which students perform at a very high level on one or more benchmarks. With this information in hand, other colleges and universities aspiring to engage their students at high levels could turn to these institutions for ideas, especially those with which they have certain features in common, such as size, educational mission, student characteristics, and other factors. We illustrate how this could be done using NSSE 2000 results for one benchmark level of academic challenge. In the absence of data spanning multiple years, we are not at this point declaring that these schools are necessarily exemplars to be emulated. At the same time, their students reported being highly engaged in Spring 2000 in these areas.
FIRST-YEAR STUDENTS Liberal-Arts Colleges Antioch College Centre College Denison University Wabash College General Colleges Columbia College (S.C.) Marymount College (N.Y.) Medgar Evers College of the City University of New York Master's Institutions College of Notre Dame of Maryland Loyola College in Maryland Regis College (Mass.) University of Richmond Doctoral-Intensive Universities Pepperdine University Polytechnic University (N.Y.) Seton Hall University Doctoral-Extensive Universities American University Indiana University at Bloomington Rice University University of Michigan at Ann Arbor Special Mission Rhode Island School of Design Rose-Hulman Institute of Technology SENIORS Liberal-Arts Colleges Antioch College Centre College Evergreen State College Sweet Briar College Wesleyan College (Ga.) General Colleges Barton College Columbia College (S.C.) Covenant College Master's Institutions College of Saint Catherine (Minn.) Regis College (Mass.) Saint Michael's College University of Richmond Doctoral-Intensive Universities Miami University Pepperdine University State University of New York College of Environmental Science and Forestry Doctoral-Extensive Universities Brigham Young University Loyola University of Chicago University of Michigan at Ann Arbor University of Virginia Note: The institutions are listed in alphabetical order and scored high on the respective benchmark for schools of their Carnegie 2000 type. A different set of institutions might result if institutional scores were adjusted based on institutional selectivity, resources, and other variables. Source: National Survey of Student Engagement, Indiana University |
The questions were clustered into five "national benchmarks of effective educational practices": level of academic challenge; the amount of "active and collaborative learning," which would include how often students made class presentations, worked on group and community projects, and tutored others; student interaction with faculty members; access to enriching educational experiences like internships and study-abroad programs; and level of campus support, gauged by factors like how much the college helps students cope with non-academic responsibilities and supports social life.
The study is one of three being developed by some of the biggest names in higher-education assessment that attempt to give consumers and colleges information that may be more useful than the rankings. The other surveys, which measure the success of freshmen and alumni, focus on many of the same institutions, and the developers of all three are looking at ways to coordinate their efforts.
In a foreword to the report of the national engagement survey, released this week, Russell Edgerton, director of the Pew Forum on Undergraduate Learning, and Lee Shulman, president of the Carnegie Foundation for the Advancement of Teaching, said that the report "marks the public unveiling of a new diagnostic tool." Pew and Carnegie sponsored the study and helped interpret the results.
"As we all know, students can be surrounded by impressive resources and yet rarely encounter classes or other activities that authentically engage them in learning," they wrote. "The N.S.S.E. 2000 Report reveals whether and how institutions are actually using their resources to provide deep, meaningful learning experiences as reported by the students themselves."
To encourage participation, survey organizers agreed to keep institutional data confidential, leaving it up to the colleges to decide whether they wanted to make their results public. The colleges that are identified in the report as standouts in each category granted permission to be named.
The report's authors found the national results both promising and disappointing.
On the whole, students at small colleges and liberal-arts colleges came out ahead in the five benchmark categories, according to the national report. Beloit, Centre, Elon, and Sweet Briar were the only four colleges that scored in the top 20 percent in all five benchmark categories for both freshmen and seniors.
"There are some really hidden jewels out there," says George D. Kuh, a higher-education professor at Indiana University at Bloomington and director of the survey. "One surprise was that some high-profile schools don't do as well as some schools that most people have never heard of."
The National Survey of Student Engagement found that liberal-arts colleges were more demanding academically, requiring more writing and more analytical thinking than other types of colleges.
But the scores varied widely within each category of institutions. Some small liberal-arts colleges did poorly in some categories, while some large, doctorate-granting universities did well.
Most students surveyed (79 percent) said that their institution expects them to study a significant amount. But less than 15 percent of full-time freshmen and seniors alike spend 26 or more hours a week studying (the traditional standard, the report says, is about 30 hours a week for full-time students). About half of the students said they spend between six and 15 hours a week, and one in 10 full-time students spends five or fewer hours a week preparing for class.
The survey also found that students at large universities are the least involved in activities that promote collaborative learning. "Arguably, these are the very institutions where active and collaborative learning approaches are most needed to compensate for the anonymity and passivity that can characterize large, impersonal learning environments," the report says.
Sixty-three percent of seniors who were surveyed participated in community service, and 41 percent were involved in a community-based project as part of a regular course. But 19 percent of first-year students never made a class presentation.
Across the board, the benchmark that most colleges struggled with was student interaction with faculty members. It was most common at liberal-arts colleges and least common at doctorate-granting institutions, where 53 percent of freshmen and 35 percent of seniors "never" discussed ideas from their readings or courses with a faculty member outside the classroom, and 79 percent of freshmen and 63 percent of seniors never worked with a faculty member in a non-academic venue.
The study also found that at the doctoral-intensive universities -- which, "given their exceptional faculty resources," the authors wrote, should have been expected to "lead the pack" -- only 17 percent of freshmen reported working with their professors on research, the lowest of any college category. At least one college that did poorly in the student-faculty-interaction category is already taking steps to improve.
The Georgia Institute of Technology has committed $250,000 to help faculty members involve more undergraduates in research projects. For example, some money might be used to send student presenters to research conferences.
Georgia Tech's score for both freshmen and seniors indicated that, on average, students work on research projects with faculty members somewhere between "never" and "occasionally," says Robert C. McMath, vice provost for undergraduate studies and academic affairs.
"The value for us in this survey was that it confirms things that we already know intuitively," he says. "At a big research university like ours, undergraduate students may have only limited contact with professors outside of class and there may be at least a perception that senior researchers are not much engaged with undergraduates."
Colleges that did well in the survey credited specific programs that promote student engagement.
Students in Elon College's general-studies program, for example, are required to participate in at least one of five "Elon Experiences": study abroad, internship, community service, leadership development, and undergraduate research. Of the students who graduated this year, half had studied abroad, three quarters did at least one internship, and two-thirds had participated in a community-service program, officials said.
In 1994, the college restructured its curriculum, adding a fourth hour to each three-credit course and dedicating that hour to "active learning" in every major. A 36-hour program in political science, for instance, would include nine four-hour courses instead of 12 three-hour courses. "It's not another hour of lecture," explains Gerald L. Francis, Elon's provost. ''Part of this would be time spent in class where students would have to work together collaboratively."
Julia Treu-Fowler, a sophomore at Beloit, was not surprised that the college fared so well. In the course of one weekend this month, she says, the college illustrated how it encourages a host of "effective educational practices" that are outlined in the survey: facilitates collaborative learning
|
Highlights From the National Survey of Student Engagement
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
She was one of about 150 sophomores who participated in an annual retreat at nearby Lake Geneva, part of a two-year support program that helps first-year students adjust to college life and helps sophomores prepare for the years ahead.
At the retreat, the sophomores, their faculty advisers, and staff members from the study-abroad and career-services offices brainstormed about choosing a major, finding an internship, managing stress, and avoiding the "sophomore slump." Each student also joined in a "hot topic" session, where participants broke up into groups to discuss solutions to campus problems ranging from tasteless dining-hall food and inadequate parking to excessive drinking and sexual assault.
Ms. Treu-Fowler chose the session on campus diversity and racial issues, where about 20 white, black, Latino, and Asian students talked about ways the campus could recruit more minorities, particularly African-Americans. One solution offered by the group was to help the admissions office by sending several students of various races and ethnicities to recruit minority students in the high schools.
"I'm white and I came from a small farming community," says Ms. Treu-Fowler, of Pullman, Wash., who is majoring in Spanish and international relations. "I have friends here from Argentina, Senegal, Turkey. That's great."
Although the survey organizers say that colleges like Elon and Beloit may potentially be identified as "engaging colleges," they are reluctant to hold them up as the standard-bearers just yet.
In the coming years, the people behind the engagement study hope that consistent high performers might emerge as models for other campuses, "but it's too premature to make that judgment now," the report says. "In addition, it would be inappropriate and unfortunate if identifying exemplars so that others can learn from them becomes the basis for another set of 'college rankings.'"
Mr. Edgerton says that the immediate goal of the survey is to allow colleges to identify their strengths and weaknesses as their students perceive them. Down the road, when colleges across the board decide to make the survey results public, he says, it could be used as an accountability tool for accreditors, policymakers, and news media. Perhaps even U.S. News & World Report will factor the survey results into its rankings, he says.
"The National Survey on Student Engagement reveals a portrait of a college that a survey of resources and reputation doesn't," Mr. Edgerton says.
Editors at U.S. News familiar with the engagement-survey concept say they are eager to see the report but doubt it could be incorporated into the rankings. "To have student input would be very valuable to us," says Anne McGrath, managing editor of U.S. News' America's Best Colleges. "But it's our understanding that the survey is limited in the number of schools it reaches and the results are not going to be made public across the board. At this point in time, it doesn't seem that there's a way we could immediately use what they're doing, other than as a potential source of strong story ideas."
As maligned as the rankings are, they are closely followed by college administrators who freely publicize the results when they do well and by high-school students in the middle of their college search. But college officials say that the National Survey of Student Engagement is more practical.
"It's nice if you're in the top 50, but I don't know what that means," says William J. Flanagan, vice president of student affairs and dean of students at Beloit College, which was ranked 47th among liberal-arts colleges in the U.S. News rankings this year. "It's more important to me that our students feel very strongly about their experiences here, and it suggests we're doing things that students really value."
Mr. Kuh, the survey's director, says there is a place for both.
"The N.S.S.E. isn't designed to be an alternative to U.S. News," he says. "It would be fascinating to put our data alongside U.S. News's and say: Given this school's reputation and how much it costs, are students engaged there? Am I getting my money's worth?"