The impact of social normative feedback on students' online learning behavior: A replication study during the Covid-19 pandemic
1. Background: Our teaching-related research before the pandemic
Before the corona pandemic, our Chair of Information Systems started to convert existing classroom courses to the blended learning format. Although we are convinced that the blended learning format offers many advantages to the course participants by enabling them to learn at their own pace or to adapt their learning plan to their personal activities (e.g., children, part-time job), we have observed that such online formats can dramatically encourage procrastination behavior. Knowing that course materials are available until the exam, individuals may postpone learning and instead focus on activities with near benefits, which could seriously harm their academic performance (Tuckman, 2005). Strikingly, there is educational research which suggests that high achievers benefit from such blended learning formats, while the performance of lower achieving students may even deteriorate (Asarta and Schmidt, 2017). It is important to note that the Covid-19 pandemic might worsen that effect, thus exacerbating educational inequality (The Economist, 2020).
To counteract these adverse effects, we started conducting field experiments that apply theories from the behavioral sciences to encourage continuous (online) learning. In the following, I will first describe the first experiment conducted before the corona pandemic, before moving on to a currently ongoing experiment.
2. First field experiment & theoretical background
To reduce procrastination behavior among the students and potentially improve their academic achievement, we evaluated a feedback intervention that focuses one of the most powerful determinants of behavior change: The activation of social norms. Individuals often orient themselves towards the behavior of others (descriptive norm) and what others approve or disapprove (injunctive norm) (Cialdini et al., 1991). As a theoretically sound evaluation of social norms was missing in digital learning environments, we integrated a social norms intervention directly into a blended learning course (elective course, master's level course). More precisely, the intervention consisted of a feedback widget / chart – placed on the landing page of the course – that showed one's study time of the previous week compared to the sample's average and to the most active students (descriptive norm). In addition, there was a smiley indicating whether the behavior was approved or disapproved (injunctive norm). An image of the social-normative feedback is attached to this post (see social_normative_feedback.png). The feedback design followed to a large extent the design of Allcott (2011) who has successfully used social norms in a study with 600,000 households to induce energy conservation.
Our first study has shown that the integration of such a feedback widget into a digital learning environment can significantly change students' learning behavior. Overall, the feedback widget increased statistically significantly the participants' online learning duration by an average of 25.4% (N = 58 individuals). When analyzing time trends, we found that the feedback was in particular effective in a time segment during the semester where the participants of the control group were relatively inactive. Here, the feedback increased the online learning time by 86.75% on average. Beyond the impact of the intervention on the learning time, we found positive effects on exam-related variables and no evidence of adverse or heterogeneous effects.
We have submitted a corresponding manuscript of this study to the ICIS 2020 and it recently got conditionally accepted. Yet, with the Covid-19 pandemic, we wondered whether our findings would also replicate under these changed circumstances.
3. Replication study during the pandemic & preliminary results
To explore whether social normative feedback has a similar impact when there is potentially less social interaction between the course participants, we are currently repeating the experiment with a compulsory bachelor's level module (N = 119 individuals): Similar to the first course, the online lectures are published weekly in the corresponding digital learning platform. There, the course participants can watch the lectures on-demand. Instead of accompanying classroom tutorials, however, the weekly tutorials are held synchronously via a separate communication platform.
The semester is almost over and we have already carried out a preliminary analysis. Interestingly, we find very similar results compared to the first study: The results indicate that the social norms have overall a large, desired effect on students' online learning time (+29.9% increase in learning time. Notably, there is no statistically significant difference to the first experiment with an effect size of +25.4%. Likewise, the treatment effect of both studies largely stems from a time segment during the semester (i.e., before the exam preparation period) in which the participants of the respective control group were disproportionally less active, probably due to self-control problems (i.e., procrastination).
4. Synthesis of the studies & outlook
Building on our first study on social norms with the blended learning format, we have successfully replicated the study during the corona pandemic with an online course, leading to two main contributions. In light of the current body of research, there is yet scarce evidence of which digital learning interventions lead to a desired real-world impact: Most of the existing studies solely investigate the influence of a certain approach by means of a single case study and therefore provide limited evidence whether scholars can support students with these approaches under different circumstances. By contrast, we evaluate the exact same feedback design by means of two university courses, thus shedding light onto the generalizability of the intervention. Second, we find evidence that the social normative intervention has still a large impact during the Covid-19 pandemic, even though there may be fewer social interactions between the course participants. Thus, our study suggests that the social normative influence is also powerful in mitigating procrastination behavior under these changed circumstances (e.g., now fully online course and compulsory course).
With the growing role of online learning in education, our experiments prompt scholars to consider the influence of social norms in the design of digital learning environments to effectively support students. The evaluated feedback requires relatively low efforts to implement (it is only a chart) and the effects are so far intriguing. Our results indicate that even those are positively influenced by the social normative feedback who use the corresponding learning platform less than the average. At the same time, we find no evidence for undesired effects due to the intervention (i.e., decreased platform visits of the learning environment). In other words: Our studies indicate that a small modification to an online learning platform can considerably support also those students with less self-regulated learning skills, who may be disproportionally affected by the Covid-19 pandemic.
As a next step of our research, we want to analyze the impact of the intervention on other important variables of interest of the second experiment (i.e., exam points, exam participation rate) and publish these findings within the IS discipline.
References
Allcott, H., 2011. Social norms and energy conservation. Journal of Public Economics 95, 1082–1095. https://doi.org/10.1016/j.jpubeco.2011.03.003
Asarta, C.J., Schmidt, J.R., 2017. Comparing student performance in blended and traditional courses: Does prior academic achievement matter? The Internet and Higher Education 32, 29–38. https://doi.org/10.1016/j.iheduc.2016.08.002
Cialdini, R.B., Kallgren, C.A., Reno, R.R., 1991. A Focus Theory of Normative Conduct: A Theoretical Refinement and Reevaluation of the Role of Norms in Human Behavior, in: Zanna, M.P. (Ed.), Advances in Experimental Social Psychology. Academic Press, San Diego, CA, USA, pp. 201–234. https://doi.org/10.1016/S0065-2601(08)60330-5
The Economist, 2020. The pandemic is widening educational inequality. URL https://www.economist.com/graphic-detail/2020/07/27/the-pandemic-is-widening-educational-inequality?utm_campaign=the-economist-today&utm_medium=newsletter&utm_source=salesforce-marketing-cloud&utm_term=2020-07-28&utm_content=article-image-4
Tuckman, B.W., 2005. Relations of Academic Procrastination, Rationalizations, and Performance in a Web Course With Deadlines. Psychological Reports 96, 1015–1021. https://doi.org/10.2466/pr0.96.3c.1015-1021
------------------------------
Sebastian A. Günther
University of Bamberg
Bamberg, Germany
------------------------------