Not Every Student Survey Deserves Coverage
Look for sample size, question bias in poll-based press releases.
Every higher education reporter gets peppered with public-relations pitches like these: Some new survey shows your local university made the list of preppiest, nerdiest, or even ‘most ratchet’ colleges.
“Most are for entertainment value,” said Steven Shepard, a Politico reporter. “They are fun and harmless.”
Finding the surveys that offer real value to readers can be a minefield for reporters, according to experts gathered at an Education Writers Association seminar in Atlanta Oct 2-3.
The key to getting poll-based stories right is to ask plenty of questions about how the surveys were collected and what questions were asked, Shepherd said. (The EWA offers a free in-depth education reporter’s guide to surveys.)
“Some are opt-in online votes,” which could bias the results, Shepard said. “And you have to make sure questions are worded in a fair way and isn’t to get a desired response.”
Verifying small or online surveys can take a lot of legwork. But at least two well-established organizations contact sufficiently large numbers of students to provide reliable information to journalists reporting on trends among college students.
Gallup, the veteran polling organization, plumbs student views in several ways and surveys:
The Gallup-Strada Education Consumer Pulse, funded by the nonprofit Strada Education Network, asks 350 adults each day questions about their higher education experiences. Add it up, and that means more than 122,000 adults are being surveyed a year, noted Brandon Busteed, education director at Gallup.
The survey will run over the next several years.
Personalized data for reporters
Of special interest to journalists: Gallup will accept reporters’ suggestions for polling questions, he said. “We will do personalized data requests,” he said. “That is a serious gesture. Our only limitation is ideas.”
Recent questions about how satisfied people are with their choice and level of education turned up newsworthy and counter-intuitive results, for example, said Carol D’Amico, a Strada executive vice president. About half of Americans would change at least one of their education decisions if they had to do it all over again, Gallup found. But among the most satisfied are those with advanced degrees and, surprisingly, she added,“we are learning that those with no degree, with a vocational path, are most satisfied.”
These kinds of findings, D’Amico predicted, will “have an effect on institutional behaviors and national and state policy.”
Additionally, the survey recently asked respondents how they got information about college. It found that most people rely on informal social networks rather than professionals, such as high school counselors.
D’Amico said the survey also will soon break data points down by state results.
Gallup also produces the Gallup Purdue Index, which asks about 11,000 college graduates about their college experiences and their current situations in an attempt to find connections between the two.
The second organization producing survey data with large, reliable samples, is the Cooperative Institutional Research Program, housed at the Higher Education Research Institute at UCLA.
The CIRP Freshmen Survey has provided a close look at the makeup of incoming college students since 1966. To date, over 15 million students at over 1,900 institutions have participated in the survey, according to the institute’s website.
Kevin Eagan, the interim director of CIRP, said the report doesn’t cover all colleges, but provides a valuable sampling of freshman behavior and demographics at four-year universities. The survey is opt-in at universities, and about 250-400 campuses participate every year.
The survey is especially helpful in viewing long-term enrollment, political and social trends. For instance, Eagan said the report looks at social issues, such as student views on marijuana or abortion over time.
Among newsworthy recent findings: The percentage of freshmen who consider themselves “liberal” has been rising since 2013 and is now the highest it’s been in more than 40 years. In fact, the percentage of females who consider themselves “liberal” hit an all-time high of 41 percent. Males, however, remain far more conservative. Only 29 percent labeled themselves as left-leaning.
Another noteworthy gender divide revealed by CIRP: Almost half of female students said they now spend at least six hours per week on social media. Only about a third of male students reported devoting that much time to sites like Twitter, Facebook or Instagram.