Course Reflection Survey FAQ

Many of the questions are specific and ask about aspects of the course that instructors have direct control over. Students are given the opportunity to respond in text boxes in even greater detail in each section of the survey, if they so choose. (Purpose 1)

Questions are designed primarily to ask students about their experiences in class, rather than to assess the quality of the instructor or instruction. This leaves it to those with expertise in the subject area and in pedagogy to draw inferences about teaching effectiveness and quality, where appropriate. (Purposes 1 & 2)

Open text boxes provide students the opportunity to provide more detailed feedback. These text boxes are framed by the previous questions and the text box prompts so as to maximize relevant and useful responses. (Purpose 2)

The TEWG believes that student feedback is important and should play an indirect role in administrative assessment of teaching. Student feedback is an element of such assessment, but we encourage departments and administrators to find ways of taking such feedback into account that does not treat it as a direct assessment of teaching in and of itself.

We will be working with pilot units to suggest ways to do this in the context of their system for evaluating teaching.  (Purpose 2)

We also think that whatever instruments are used by administration for evaluation purposes should be as free from the effects of implicit (and explicit) bias as possible. The questions in the Course Reflection Survey were crafted with concerns about implicit bias at the forefront, based on up-to-date research on the prevalence and mitigation of such bias. (Purpose 2

Models and motivations

We used insights from work and discussions at other institutions or organizations, including:

1. University of Oregon  https://provost.uoregon.edu/revising-uos-teaching-evaluations

2. Teaching Quality Framework at the University of Colorado and other affiliates   https://www.colorado.edu/teaching-quality-framework/about-tqf

3. AAU report “Aligning Practice to Policies: Changing the Culture to Recognize and Reward Teaching at Research Universities”  https://doi.org/10.1187/cbe.17-02-0032

4. National Academies: National Dialog on Transforming STEM Teaching Evaluation in Higher Education  https://www.nationalacademies.org/event/01-14-2021/national-dialogue-on-transforming-stem-teaching-evaluation-in-higher-education

5. The TEWG collected survey data that include how faculty responded to the old STE system.  (We plan to collect data that compares faculty responses to the new Course Reflection Survey.)
   

Evidence that we are on the right track

As suggested by a meta-analysis of over 100 surveys on improving teaching surveys, there are concrete steps that universities can take and the OU survey is an example of how many of these steps can be actualized. For example, the survey is designed not to evaluate teaching, but to focus on the perception of a student’s experience in their class. To provide for more useful feedback, we’ve restructured questions so that we are not relying on qualitative comments as a measure of a student’s learning experience. TEWG and CFE has also developed resources to support interpretation of the Course Reflection Survey and also make recommendations to improve student response rate.

1. The Center for Faculty Excellence has developed a document to help instructors understand and process their feedback.
   
   Power of Their Words - Faculty
   Power of Their Word - GTAs
   
   

2. We recommend that you talk with your departmental and disciplinary mentors/advisors/colleagues about your feedback to gain discipline-specific insight into your Course Reflection Survey responses. 
   
   

 

3. In the future, the Graduate College will also be offering a series of workshops focused on teaching that may include reading, responding to, and integrating feedback from students in your future teaching.
   
   

 

4. Dr. Hong Lin, Senior Faculty Development Specialist at the CFE, is also available to meet with faculty to provide additional support.

1. Explaining how you use the Course Reflection Survey feedback and why it is valuable can help motivate students. 

2. Giving them dedicated time in class to complete it encourages the highest participation rates. 

3. You may consider giving a small amount of extra credit if a certain percentage of your students fill out the Course Reflection Survey, e.g., “If 80% complete the Course Reflection Survey, everyone gets 5 extra points.”  You can track the response rate during the open period at https://coursereflection.ou.edu.

All instructors that are officially listed as instructor-of-record in Banner and have a percentage of responsibility of greater than 0% will be included in the survey. 

1. Ask your chair/committee A how they will be incorporating information from the Course Reflection Survey into instructor teaching evaluations. If they do not currently have a rubric for doing so, ask if there is a plan to develop one.
   
   

2. Invite TEWG to present at a faculty meeting.

Evidence shows that responses to general questions like those are not indicative of good teaching practices. In other words, they don’t reflect anything about your actual teaching effectiveness and they reward and punish instructors for traits that have nothing to do with how good a teacher they are, which is unfair.  It is true that having such questions on a traditional survey provides a convenient and concise numerical result to put on job applications, but it does not actually measure what it claims to.  We encourage you to reflect upon your CRS feedback and include information in your job applications about how you used the feedback to improve your teaching.

The University of Oklahoma Norman campus collects students' feedback and perspectives about their course experiences using the Course Reflection Survey. The Course Reflection Survey is not a traditional Student Teaching Evaluation (STE) and does not include quantitative evaluative metrics. Instead, the Course Reflection Survey provides distributions of student responses to a wide range of questions and qualitative feedback that instructors can use to reflect upon, evaluate, and adjust their teaching methods and course materials. This change reflects a growing consensus, based on a wide range of empirical results, that traditional, quantitative STE’s are inaccurate in reflecting the quality of one’s teaching.  

1. There will soon be examples of such rubrics posted on the Course Reflection Survey website to give your department a place to start the discussion.
   
   

2. TEWG is planning to hold some workshops for units in the fall to discuss their annual teaching evaluation rubrics and how Course Reflection Survey information and faculty self-reflection can be incorporated.
   
   

3. TEWG recommends that departments not use the Course Reflection Survey directly in annual evaluations, but instead evaluate faculty members based on their incorporation of Course Reflection Survey feedback (and other input) toward improving their course.  Of course, other factors will also contribute to teaching evaluation.