Tapping Public Opinion Polls to Strengthen Stories
We asked some of the education reporters who joined us at Stanford University in May to contribute blog posts from the sessions. Today’s guest blogger is Mark Walsh of Education Week.
Public opinion research is seemingly everywhere these days, from the sophisticated polls that drive politics to the less-scientific quick hits from SurveyMonkey and other offerings on the Web. What do trends in public opinion surveys mean for education policy and the reporters who cover it?
The participants in a session on using polling data in reporting noted that just a day earlier, many attendees at the Education Writers Association’s national conference received a news release about a poll of teacher views on one of the hottest topics of the conference, the Common Core State Standards. The American Federation of Teachers surveyed 800 of its members and found that three-quarters supported the Common Core standards for mathematics and English language arts, but just 27 percent said their school districts had provided them with the tools and resources necessary to teach the standards.
For Regina A. Corso, a senior vice president with Harris Interactive, the AFT survey was a good starting point to remind reporters about some of the basics of analyzing polls. Who performed the survey—a reputable polling organization or some other entity? Who was surveyed and how large was the sample? A sample of 1,000 individuals is typical for most national polls, Corso said, “and if they are sampling correctly, that sample size is going to be valid.”
Reporters should also examine when the survey was performed, the margin of error, and other aspects of the methodology. Red flags should go up if the entity releasing the survey won’t disclose the methodology or the survey questions. Perhaps most important is to know who commissioned the poll or survey, and to keep that in mind when examining the sponsor’s own releases about the results.
“Of course a survey report will have spin” from the sponsor, Corso said. “But as long as the questions are available and the report isn’t cherry-picking the data,” the survey report is likely valid, she added.
[The AFT survey on the Common Core standards turns out to have been conducted by Hart Research and Associates, which appears to have quite a few education clients.]
Brandon Busteed, the executive director of education polling for the Gallup Organization, told the breakout session that education is a frequent subject of polling efforts at Gallup, from specific client work to questions regularly inserted into the organization’s nightly U.S. polls and even its World Poll. Gallup is regularly sampling 98 percent of the world population, Busteed said, with North Korea and Papua New Guinea the only nations missing from that global effort.
Gallup has convened a K-12 superintendents’ panel, with the first poll results from that group coming soon, Busteed said. Meanwhile, the organization has a college presidents’ panel and a March report released in conjunction with the online news site Inside Higher Ed found that campus chiefs were “not enthusiastic about MOOCs,” Busteed said, referring to the “massive open online courses” that also attracted much analysis and discussion at the EWA meeting. The survey shows that only 14 percent of college presidents strongly agreed that MOOCs have “great potential to make a positive impact” on higher education, with 28 percent of respondents agreeing with that statement, while 31 percent disagreed or strongly disagreed, and the rest were neutral.
Busteed highlighted an example of recent Gallup polling data which could be mined for a perspective on job satisfaction among teachers.
“Teachers have the second highest well-being of any career,” Busteed said in reference to Gallup-Healthways Well-Being Index, which Gallup itself turned into an education-specific news release. The index showed that teachers trailed physicians but beat business executives, nurses, sales professionals and other job categories.
But teachers are more in the middle of the pack in ranking their work environments, and they came in last among 14 job categories in the survey in agreeing with the statement that “my supervisor always creates an environment that is trusting an open,” Busteed noted.
The latter result prompted certain wags in this breakout session of education reporters to ask where journalists ranked on that question. Busteed noted that journalists were not included as a job category in the well-being index, likely because there was not a statistically significant sample response.
The bottom line is that the major polling organizations do a lot of work for education clients as well as general survey questions about education and that much of the data is there for use the enterprising reporter. Corso and Busteed also said that their organizations are usually willing to put forth their own experts to help explain results.