New online tool developed by Stanford researchers helps schools spot struggling readers in a fraction of the usual time

A pandemic workaround takes off

The ROAR actually originated as a COVID-19 research workaround for the lab, where Yeatman and his team study the mechanisms in the brain that underlie learning.

One focus of the lab is to develop interventions for young children with dyslexia, research that relies on brain imaging studies to observe the effects an intervention has on the brain. That work includes developing targeted measures to evaluate a child’s growth in specific reading skills.

“All of this requires working in close, one-on-one or small group settings with children,” said Yeatman. “When the pandemic hit, with all of its uncertainty, we realized that we needed to come up with some new innovations in order to continue pushing our research forward.”

The researchers thought an online assessment might function reasonably well in place of the standardized in-person version they had been using, and they developed a prototype of the ROAR. Subsequent validation studies showed a remarkably high-reliability rate: Scores generated by the ROAR correlated strongly with those from standardized in-person screenings, including the Woodcock-Johnson assessment of basic reading skills, widely considered the gold standard.

“The correlation was almost perfect, meaning these measures are tapping into a very similar construct,” said Yeatman. “Honestly, we didn’t expect it to work as well as it did.”

Meanwhile, Yeatman and his team had been working with local school officials around new California guidelines issued to help teachers identify and support students with dyslexia. That relationship – developed through the Stanford-Sequoia K-12 Research Collaborative, a partnership launched in 2018 to connect GSE researchers with San Mateo County school districts to explore challenges facing the districts – laid the groundwork for helping to pilot and refine the ROAR.

“We’d had a lot of conversations [with Yeatman] about how we could get more data, where we could find a good universal screener, how to go about implementing the guidelines – we learned a lot from him and his team,” said Marta Batlle, student services director at the Woodside School District, one of the districts in the collaborative. “When they asked if we wanted to partner to administer the ROAR, we were really interested in having more data, and we wanted to be part of the research process.”

Making a game of it

The ROAR, which runs on any web browser, launched with an assessment that measures students’ ability to quickly recognize words, a foundational skill for reading fluency and comprehension. It’s designed to feel like a computer game, with cartoon-like characters walking students through the activity and encouraging them along the way. A mix of actual and made-up words flash on the screen; the student’s task is to determine whether each word is real or not.

“The kids love it,” Batlle said. “They’re engaged, and they want to do it.”

Students as young as first-graders can do the assessment on their own, in the classroom or at home.

The lab collaborated closely with school leaders to develop interactive score reports for the educators, providing statistics and visualizations from the district level down to individual classrooms and students.

“The reports provide immediate, actionable data back to reading specialists and teachers, right after students have taken the ROAR,” said Amy Burkhardt, director of research and partnerships at the Brain Development & Education Lab. (The scores are stripped of any identifying student data for Stanford’s use.)

“With the ROAR, we got a measure for every student in the classroom in about the same amount of time it takes us to administer our usual screener with one child,” said Ching-Pei Hu, assistant superintendent at the Belmont-Redwood Shores School District, another district in the Stanford-Sequoia K-12 Research Collaborative that worked with Stanford to pilot the ROAR. “It’s fast, it’s efficient, and it was reliable in giving us a baseline of who we need to dig into some more.”

Looking ahead

As they work with schools and clinics to refine the ROAR’s tool for assessing single-word recognition, Yeatman and his team are developing others for more complex reading skills, including phonological awareness (the ability to recognize and manipulate sounds) and the ability to silently read and understand full sentences quickly and accurately.

Through pilot funding from Stanford’s Office of Community Engagement, the team recently expanded their research collaborations to include KIPP Schools of Northern California. The lab is also collaborating with the Developmental-Behavioral Pediatrics clinic in the Stanford School of Medicine to explore the use of these assessments as a screener for children with learning differences who enter Stanford Hospitals and Clinics.

Beyond providing a practical tool for clinicians and educators and a better understanding of learning differences, the researchers hope the data generated by the ROAR will ultimately help in developing tailored approaches to help struggling readers.

“You can see kids from the same background – the same families, even – entering kindergarten, where they learn how letters can be combined to make words and represent human language,” said Yeatman. “Some kids dive right into that and they’re suddenly reading, but others struggle deeply for many years to learn this mapping, to make it automatic, to be able to use written language as a fluid form of communication. We’re trying to understand the difference – and the hope is that as we begin to understand the barriers for some kids, we can develop targeted interventions.”

The Rapid Online Assessment of Reading (ROAR) is among the projects of the initiative on Learning Differences and the Future of Special Education at Stanford Graduate School of Education. The Stanford Transforming Learning Accelerator, which creates and delivers new learning solutions, helped facilitate the use of the ROAR with schools and other partners.

Early funding for the tool was provided through a Neuroscience:Translate award from the Wu Tsai Neurosciences Institute at Stanford.