Stanford-led study finds advantage in more scientific instruction to improve compliance with COVID-19 guidelines

It’s a question at the heart of many public health awareness campaigns: Is it better to stick to a few basic facts and keep the message short, or to risk overwhelming your audience by explaining the science behind the guidance?

A new study led by Stanford researchers compared the approaches by testing the impact of two different interventions encouraging compliance with COVID-19 health guidelines: a short video conveying basic facts about COVID-19, and a longer video that included the same facts along with explanations of underlying scientific concepts.

While both videos improved participants’ knowledge, attitudes and behavior related to the disease, the longer video proved more effective at sustaining the improvement than the shorter, facts-only version.

“There’s an idea that shorter and more concise statements are better,” said Prashant Loyalka, an associate professor at Stanford Graduate School of Education and one of the study’s authors. “It may seem counterintuitive that a longer intervention would be effective. But we’re seeing that when people understand the underlying scientific reasoning, that’s when it really sinks in.”

Adding context to guidance

The researchers set out to compare the different approaches in April 2020, soon after COVID-19 was declared a global pandemic and information campaigns gained urgency. 

“We noticed that a lot of the information that was being provided both in the United States and around the world was typically very simplistic: Wear a mask, wash your hands, don’t go out,” said Dinsha Mistree, a research fellow and lecturer at Stanford Law School and the study’s lead author. “There wasn’t often a lot of context delivered about why.”

Finding little evidence in the field comparing the effectiveness of these different approaches, the researchers quickly organized a randomized controlled trial with thousands of low- to middle-income youth in urban India, which currently has among the highest number of confirmed COVID-19 cases in the world.

To recruit study participants, Mistree and Loyalka reached out to the Freedom Employability Academy, a nonprofit organization they’d worked with in the past, which provides free employability-skills training to young people ages 15-27 throughout urban India. “We targeted the interventions to this population, young people who are in a lower socioeconomic strata and just learning English,” said Mistree. “We couldn’t assume they knew much about viruses or biology.”  

They developed the videos with help from three high school students in San Jose, one of whom had done an internship at Stanford’s Freeman Spogli Institute for International Studies. (The team of three students recently won the grand prize in a national student documentary competition run by C-SPAN, for a video they created about technology’s damaging effect on democracy in 2020.) The researchers also worked with a video production company in India to finalize the materials, ensuring that the content was culturally appropriate.

Sustained improvement over time

Study participants were randomly assigned to watch either a 10-minute video conveying basic facts about COVID-19 – common symptoms, how to protect yourself and others – or a longer video that included an explanation of scientific concepts behind the guidance, including coronavirus biology, asymptomatic spread and exponential growth. A third control group watched neither.

After their assignment, all participants took three surveys over the course of the following month: one immediately after the intervention, a second one week later and a third about two weeks after that. The surveys measured the students’ knowledge (including applied knowledge, or their ability to extend an understanding of facts to different situations); attitudes, such as whether they believe personal actions can make a difference in the spread of COVID-19; and behavior, such as mask-wearing and social distancing. 

The surveys taken immediately after the intervention showed increased knowledge, attitudes and behavior among students who watched either video, compared with the control group. But participants who watched the longer, more explanatory video showed more sustained improvements over time.

“Instead of reducing their attention, the longer video actually did increase their understanding,” said Mistree. 

The findings suggest added value in explaining underlying scientific concepts, said Robert Fairlie, a professor of economics at the University of California at Santa Cruz and a coauthor of the study. “Perhaps the ‘why’ is a needed addition for compliance.”

The researchers acknowledge that one possible limitation of the study is its reliance on self-reported data, rather than objective measures of attitudes and behaviors. “It’s just not feasible for us to track things like mask-wearing and proper handwashing, especially at this scale,” said Mistree. But because subjects were randomly allocated into the different intervention groups, he noted, any degree of misreporting would likely be equally distributed and therefore not bias the findings. 

The researchers hope their findings will help government and health agencies worldwide design and deliver effective instructional materials for young people, especially as COVID-19 continues to spread and health experts warn of additional waves. They are also working to to apply their findings toward an instructional intervention explaining vaccine safety and encouraging compliance.

The study’s other authors are Ashutosh Bhuradia of the Harvard University Graduate School of Education; Manyu Angrish of the Freedom Employability Academy; Jason Lin, Amar Karoshi and Sara J. Yen of the Harker School; Jamsheed Mistri of the University of California at Berkeley; and Vafa Bayat of Bitscopic, Inc.