1:01:47
Best Practices in Survey Governance
UC Berkeley Center for Studies in Higher Education
Overview
This video discusses the critical need for and implementation of survey governance in higher education institutions. Panelists from various universities share their experiences, challenges, and best practices in managing the increasing volume of surveys, combating survey fatigue, and improving data quality. Key themes include gaining institutional buy-in, establishing coordination committees, leveraging technology like Qualtrics, and the ongoing effort to balance centralized oversight with institutional autonomy. The discussion highlights the shift from a 'sticks' approach to 'carrots' and emphasizes collaboration, communication, and the strategic use of existing data to enhance survey effectiveness and reduce respondent burden.
How was this?
Save this permanently with flashcards, quizzes, and AI chat
Chapters
- The increasing volume and ease of conducting online surveys have led to challenges like decreased response rates and survey fatigue.
- Survey governance is essential for institutional awareness, procedural structure, and promoting better survey methodology.
- The goal of survey governance is to obtain better, representative data, increase participation, and reduce respondent burden.
- A poll indicated that 39% of attendees' institutions have a survey coordination group, 39% do not, and 21% are exploring the idea.
Understanding the growing problem of survey overload and the purpose of survey governance sets the stage for why institutions need structured approaches to manage surveys effectively.
The proliferation of online surveys, where users are immediately asked for feedback after transactions, is likened to junk mail for mail-in surveys in the past.
- Institutions beginning survey governance efforts focus on describing the problem (proliferation, decreased response rates, overlapping efforts) to gain buy-in.
- Steps include reviewing current informal practices, consulting with similar institutions, and drafting policies with input from various campus partners.
- A 'survey management plan' is preferred over a 'survey approval policy' to sound less intimidating and encourage collaboration.
- Piloting initiatives like a 'student opinion panel' with opt-in participation and incentives can offer an alternative to broad, random sampling.
Successfully implementing survey governance requires strategic efforts to secure support from leadership and stakeholders, framing the initiative in a way that highlights benefits rather than restrictions.
UC San Diego is developing a draft executive summary of campus policy and sharing it with partners in student affairs, assessment, and resource administration to gather feedback.
- Established survey governance groups, some existing for over a decade, rely on a sponsor (e.g., vice provost) to champion the cause.
- Key strategies include creating annual plans, establishing committee structures with broad representation, and fostering strong partnerships with departments like IT.
- Leveraging technology, such as consolidating survey platforms (e.g., Qualtrics) and using purchasing data reports, helps centralize efforts and demonstrate value.
- Continuous communication, education for users, and creating accessible online resources are vital for ongoing engagement.
Long-term success in survey governance depends on sustained effort, strategic partnerships, and adapting strategies to maintain momentum and institutional support.
Northwestern University has had a Student Surveys Planning Group since the 1990s, with an executive committee and a larger group meeting regularly to share results and plan.
- Procedures to protect students from being oversurveyed include sample management, where a central entity draws samples to avoid repeated contact.
- Institutions often lack 'sticks' (enforcement) and rely on 'carrots' (benefits, collaboration) and goodwill to ensure compliance.
- Denying surveys that lack a clear business or educational need and directing requesters to purchase samples or use other vendors is a common tactic.
- Standardized constructs and definitions across surveys improve data comparability and reduce redundant data collection.
Effective survey governance balances the need for institutional data with the ethical responsibility to protect respondents from excessive surveying and to ensure the data collected is meaningful and reliable.
At Purdue, a central entity manages sampling for approved surveys, ensuring no undergraduate student was sampled more than four times in a recent semester, while tailoring samples to researcher needs.
- Survey governance structures are increasingly connected with data governance, using common definitions and data layers to ensure consistency.
- Success is often measured by intermediate outcomes like better-constructed surveys, improved item quality, and increased use of existing data, rather than immediate response rate increases.
- Stopping inappropriate surveys and spreading the survey burden more broadly among students are tangible signs of success.
- Fostering a culture where leaders consult survey governance groups before planning surveys indicates ingrained institutional practice.
Integrating survey governance with data governance and focusing on both intermediate and long-term outcomes provides a comprehensive view of success and ensures that survey efforts align with broader institutional data strategies.
Northwestern University's survey group is considered successful when leaders ask, 'Have you asked U-SAT when they can schedule that?' indicating the group is integrated into the planning process.
Key takeaways
- Survey governance is crucial for managing the increasing volume of surveys, combating respondent fatigue, and improving data quality.
- Gaining institutional buy-in requires framing survey governance as a collaborative 'management plan' that offers benefits, rather than a restrictive 'approval policy'.
- Successful survey governance relies on strong leadership sponsorship, cross-departmental collaboration, and effective communication strategies.
- Leveraging technology, such as centralized survey platforms and sample management databases, is key to efficient oversight.
- Institutions often use 'carrots' (incentives, improved services, data access) rather than 'sticks' (punishments) to encourage compliance with governance policies.
- Focusing on intermediate outcomes like improved survey design and increased use of existing data is a more realistic measure of success than immediate response rate increases.
- Integrating survey governance with broader data governance ensures consistent definitions and appropriate use of sensitive data.
Key terms
Survey GovernanceSurvey FatigueResponse RatesInstitutional ResearchData GovernanceQualtricsSample ManagementSurvey CalendarStudent Opinion PanelStandardized Constructs
Test your understanding
- What are the primary challenges that necessitate survey governance in higher education institutions?
- How can institutions effectively gain buy-in from stakeholders for implementing survey governance policies?
- What strategies do universities employ to sustain their survey governance efforts over the long term?
- How do institutions balance the need for survey data with the ethical responsibility to protect respondents from over-surveying?
- What are the key indicators of success for a survey governance program, and how do they differ from immediate results?