Revolutionizing dyslexia screening: Technology to empower teachers

Jason Yeatman (00:00):

Technology should be there to support the teacher, and I think assessment is this perfect place.

Denise Pope (00:08):

Welcome to School's In, your go-to podcast for cutting-edge insights in learning. From early education to lifelong development, we dive into trends, innovations and challenges facing learners of all ages. I'm Denise Pope, senior lecturer at Stanford's Graduate School of Education and co-founder of Challenge Success.

Dan Schwartz (00:31):

And I'm Dan Schwartz. I'm the dean of the Graduate School of Education, and the faculty director of the Stanford Accelerator for Learning.

Denise Pope (00:41):

Together we bring you expert perspectives and conversations to help you stay curious, inspired, and informed.

(00:50):

Hi, Dan.

Dan Schwartz (00:51):

Denise, it's always good to be with you.

Denise Pope (00:54):

How you doing?

Dan Schwartz (00:55):

Well, I'm good. So here's your question of the day. So it seems to me there are certain things with young children that you'd kind of like to know early because it's going to affect their abilities to learn going down the line. So it seems like glasses, do kids need glasses? But I kind of heard all these stories that nobody discovered this kid needs to sit in the front of the room. Is there a history to them sort of finally saying, "Gee, we should measure whether kids can see"?

Denise Pope (01:26):

Yeah, no, I know, a lot of times you don't find out until it's too late. And I know there are things that you want to catch early on because you can do some interventions early on that maybe won't be as effective later. So I think glasses, hearing, yeah.

Dan Schwartz (01:39):

So here's one, reading. So reading's important for everything to keep learning and it's kind of a tough space, because how do you measure reading before they've had a chance to learn to read?

Denise Pope (01:51):

Oh, total catch-22.

Dan Schwartz (01:54):

Yeah. So we have the guest who's going to answer this. So we're really pleased to have Jason Yeatman back. He's a professor in three separate departments at Stanford. He's very Stanford. The Graduate School of Education, the Department of Psychology, and he's in Developmental and Behavioral Pediatrics in the School of Medicine. Jason is very busy. He's a neuroscientist, but he is the neuroscientist of behavior, of how people behave, and one of his areas is reading. It's an important area. And he developed a new tool called ROAR or the Rapid Online Assessment of Reading. And he's going to explain to us how he manages to measure people acquiring a skill before they've acquired it. So welcome, Jason.

Jason Yeatman (02:40):

Good to be here.

Dan Schwartz (02:42):

Is it the case that we used to measure reading at the end of third grade was the first time that we'd actually do some formal assessment of reading?

Jason Yeatman (02:51):

Yeah, and that's something that's changed a lot now. I think either 42 or 43 states have universal screening legislation. This is legislation that mandates screening all students for dyslexia. So dyslexia, which is a developmental challenge in learning to read. And this universal screening legislation usually requires screening a couple of times a year starting in kindergarten through grades one and two. And then now getting to this grade three you're talking about, that's usually when state testing starts. So the idea is that students are screened for reading difficulties right away and for the underlying factors that are going to predict future reading difficulties. This has been a huge legislative achievement in trying to bring more equity to screening.

Dan Schwartz (03:34):

Can I ask a tangential question for a second?

Jason Yeatman (03:36):

Only if I'm allowed to give a tangential answer.

Dan Schwartz (03:40):

If you follow my tangent, yes. So dyslexia is not a thing, right? Dyslexia just means bottom 5% of readers and then there's different types of dyslexia or are all dyslexias the same thing?

Jason Yeatman (03:53):

Yeah, so dyslexia refers to a challenge learning to read. Amongst people with dyslexia, there are a number of different contributing factors. And so when you're thinking about screening, you said this was a tangential question, but actually wasn't, it's core to your first question.

Denise Pope (04:07):

I know. I think it's totally on topic and very important.

Dan Schwartz (04:09):

Yay, yay.

Jason Yeatman (04:10):

Yeah, so when you're thinking about screening for dyslexia, what you're trying to do is tap into these various mechanisms which are developing early on. They're developing before a kid learns to read. So language skills broadly are one factor that contribute to learning to read. First you establish various aspects of spoken language, and then upon that foundation of spoken language, you start layering on written language. And that's the job of school is to start building up written language on top of your spoken language skills. But kids that are struggling for a variety of reasons with spoken language, it's going to be harder learning to read.

(04:47):

But there's more detailed factors as well. So most well-known one, the one that's written into pretty much all of these policies across all the states that have universal screening is measures of phonological awareness. Phonological awareness refers to your ability to hear that speech is composed of constituent sounds. Take the word cat, which is composed of K-æ-T, I can hear those as separate sounds. And if you aren't able to attend to the sound structure spoken language, then the idea of saying, "Well now, Dan, take K-æ-T, and assign a visual symbol to represent each of these sounds," that would just make no sense to you. So phonological awareness is one of these foundations where you can really see the clear connection, right? As I explained that like, oh yeah, of course, if you can't break down speech into its constituent sounds, then how can you learn to represent these sounds with visual symbols?

Denise Pope (05:41):

Okay, so this is super fascinating. If I have a kid in kindergarten and they don't know how to read yet, you're telling me that you can do some kind of measure that will predict that they're going to have a hard time reading, am I understanding that right? Is that right?

Jason Yeatman (06:01):

Yeah, exactly. Yeah, so phonological awareness, this is one thing you can measure. Actually, I have a five-year-old, just turned five-year-old at home, she doesn't know how to read yet. She's still in preschool, one of the wonderful preschools here on campus. And as someone who studies reading, I'm starting to play these games with her of looking at her ability to attend to the sounds in language and looking informally at these different risk factors that I know are going to predict reading development.

Denise Pope (06:27):

Give an example. Do you say, "K-æ-T, what does that mean?" Give an example of how you do this. Maybe not with your own five-year-old, but in real life.

Jason Yeatman (06:36):

No, I'll be reading her a book and ask her if she can tell which words start with the same sound. She doesn't know all the letters, it's not looking at the text. But I can say, "Today three people were on a podcast together, Jason, Denise, and Dan, which of these names starts with the same sound?" The answer is Denise and Dan, they both start with a 'Duh'.

Denise Pope (06:58):

That's so great. I actually would not have thought about that. And if they can't figure that out yet, how do you know they're just not ready to read? How do you know that it's not like a challenge versus they're just not ready? Because I had a kid who didn't start reading until he was kind of end of first grade.

Jason Yeatman (07:17):

Now you're getting into the measurement question. So now we're going from... I'm going to restate your question, you're saying, "Jason, how do you go from you and your daughter hanging out and playing games to something that works for a system, a school system where we want to not just assess Jason's kid and Denise's kid, but we want to universally screen all the kids that are entering all the California kindergarten classrooms?" And that's where you build measures. So within ROAR, Rapid Online Assessment of Reading, we have a phonological awareness measure that looks at your ability to hear the first sounds in words, hear the last sounds in words, segment the middle sounds and words. So kind of different aspects of phonological awareness, your ability to attend to rhymes. So there's a suite of phonological awareness measures. And we've now had tens of thousands of kids around California, but all around the country, now ROAR is used I think in either 29 or 30 states.

Denise Pope (08:10):

Congrats.

Jason Yeatman (08:11):

So that allows us to collect normative data and make sure that the phenomenon I'm talking about isn't just some unique phenomenon to like, the kids that I see around Stanford, but really reflects typical development across the incredible diversity of learners around the United States. And then based on that, we start running studies of predictive validity. So we have a scientific hypothesis, this work started in the mid-late '80s, like the idea that phonological awareness is a key foundation. So researchers started developing measures to tap into phonological awareness, started running studies to look at how these measures collected before a kid starts school, predict their reading into the future. And now we've built on that and we've built ROAR to this whole automated platform, which rather depending on me as a researcher going and administering measures to each kid, we can have, for example, all the incoming kindergarteners in California log in, take our ROAR measures, and we can come back a year later and measure the reading skills and look at how these measures predict each other.

Dan Schwartz (09:11):

So pause there for a second. So the way we used to do it was it was a one-hour in-person assessment with third graders with an adult in the room administering it. Now I have a kindergartner in front of a computer, how do you make sure they're actually looking at the screen and that they don't drop off out of boredom?

Denise Pope (09:35):

Right, or they're dogging it.

Dan Schwartz (09:37):

Right. So while that's a great challenge, how do you make an assessment that people want to take?

Jason Yeatman (09:42):

Yeah, that's the challenge. This intersection, the part that's fun and fulfilling is this intersection between design and thinking about gameplay and child-centered design and designing technology for kids is a challenge in its own, right? You have to have something that's engaging, that's fun, that's intuitive. But there's also a science to that too, right? ROAR started off as a pandemic project, but since then it's grown where each aspect of ROAR were- for example, have a hypothesis about how an aspect of gameplay is going to improve the fidelity of the data we get by keeping students more engaged. We can run that now as a randomized control trial and experiment where we randomly give different participants different versions of ROAR and we study how it affects the engagement, how it affects the reliability of the measure. These are all aspects-

Dan Schwartz (10:38):

What's the answer?

Jason Yeatman (10:40):

Yes.

Dan Schwartz (10:40):

Yes?

Denise Pope (10:42):

Some kids... This is my hypothesis, the kids who have the fun version are going to stay more engaged than the kids who have the not fun version. Is that a fair hypothesis, Jason?

Jason Yeatman (10:53):

Yeah, and it turns out to be true. So actually I was joking around a little bit, but yeah, no, this is actually a serious study we did. Because as you start gamifying things, you come to decisions about giving people trial by trial feedback. One of the thing about gameplay that's engaging is that you know when things are going well versus not.

Denise Pope (11:12):

The player knows.

Jason Yeatman (11:14):

Yeah, yeah. Exactly, right?

Denise Pope (11:15):

Yeah.

Jason Yeatman (11:15):

So you're getting feedback, that's part of what keeps you engaged in a game, which is different than a normal assessment where, you know Dan, you were talking about delivering in-person assessments, and when you're trained on these as a teacher, you're trained to not give any feedback, to reward the fact that they're listening to you, but not to give any indication. And so we ran a study looking at how trial by trial feedback affects performance and there's different camps of teachers that had worries different ways. So we work with about a half dozen schools that specifically serve children with dyslexia. And in those schools, some of the teachers were worried that getting feedback that they'd gotten answers incorrect could be just emotionally hard for the kids, or that getting feedback that they're getting them correct would allow them to game the system in some way.

(12:05):

So this is why it's really important to actually study this rather than just sticking with an old approach of saying, "Okay, well rather than, there's all these worries that come when you start gamifying things, let's just not do it," right? That's one way we could. We're going into this, it's studying each step, and Denise is correct, adding in trial by trial feedback increases kids' engagement, they provide more useful data, they stay focused longer, and it doesn't lead to any kind of gaming the system. There are going to be some portion of students where you don't know if their low score was because they really were struggling with it or because they weren't putting in their full effort. And that's an issue whether it's delivered one-on-one or through a platform. And it turns out you engage different kids different ways, and in many ways I spend a lot of time developing technology, but I also think technology should be there to support the teacher.

(12:59):

Technology should be there to support the teacher, and I think assessment is this perfect place. You think about, Dan, if you're running a school district and you have a limited amount of time for professional development with your teachers, I would prefer that you spend that time working on strategies to intervene and improve kids' reading skills as opposed to, you know, hammering away on how do you deliver this assessment in compliance with the technical manual. And that's where technology can lift up teachers, right? Through this automated approach where we can study the different aspects, gamify, get a standardized measure, it allows teachers to spend more time doing what teachers want to do, which is teaching.

Dan Schwartz (13:43):

So Denise, when I taught English or language skills, I often thought kids who weren't doing well the problem was motivation.

Denise Pope (13:53):

What grade are you teaching?

Dan Schwartz (13:53):

Seventh, eighth.

Denise Pope (13:54):

Seventh, eighth grade, okay.

Dan Schwartz (13:56):

Yeah, and I've decided that the problem's motivation. So I do a lot of work to try and create the reading activities be more meaningful, more interesting. But I had no way to know that kids were having trouble sounding out letters. I never had kids read aloud in class. I didn't like to do that. Did you have this kind of problem when you taught?

Denise Pope (14:18):

Yeah, I mean, I don't think you should have kids read aloud in class in some sense exactly for that reason, right? Well, I'll tell you here, this is how my husband was taught history. I'm not making this up. Okay? You would come in and you would start with the first person at the front of the row and you would read the first paragraph of the history book, and then the second person would read the second paragraph and the third person would read the third paragraph and he would count ahead to know what paragraph that he was going to read to see if there were any big words or hard words or whatever, and get kind of practice. And if anyone was out of line or whatever, he was totally, totally toast, right? He'd practiced the wrong paragraph. And if you got lost or you didn't know where you were, you had to stand up, you had to stand up for the rest of the class. So it was very much this very scary... Imagine if you had dyslexia-

Dan Schwartz (15:10):

You had to stand up for rest of the class, what theory is that?

Denise Pope (15:14):

This is punishment for not paying attention because, same thing, the teacher did not understand that it was a reading struggle or you didn't want to read out loud or maybe you were embarrassed by an accent or whatever, right? The teacher was just pissed that you messed up the system and you weren't paying attention.

Dan Schwartz (15:34):

Wow.

Denise Pope (15:35):

I know.

Dan Schwartz (15:35):

Sorry. Just the idea that you would've someone stand up so that they're both physically and socially uncomfortable.

Denise Pope (15:41):

Yeah, talk about a total degradation ceremony, right?

Dan Schwartz (15:44):

That's wrong.

Denise Pope (15:46):

But I agree with you. I would never have students read out loud, and I will tell you this, sometimes it's a play, like Romeo and Juliet, where it is actually kind of important to have someone read out loud or to hear it read out loud. And you have to be careful because you don't know what's going on with those kids. You don't know who's going to be comfortable and not comfortable and stumble over words or whatnot. You really don't want to shame kids, particularly when it comes to anything in the classroom, but reading especially. But yeah, I would have no idea. I would think it was motivation too, right? If you have no idea, and by the way, we weren't trained to assess reading skills, were you? I wasn't.

Dan Schwartz (16:25):

No, no.

Denise Pope (16:27):

Never. Never.

Dan Schwartz (16:31):

So Jason, something I've seen about computerized assessments is they can be a lot more precise than teacher's ability to develop instructional interventions, right. And so isn't there a model where the computer does the assessment and then does the treatment as well and the teacher's out of the loop?

Jason Yeatman (16:53):

You know, I think humans learn best from other humans. Social learning is hugely important, and I think we often think about the goal of school of just filling the brain with more things. And it's important, at some level, if you were to tell me that my kid was going to go to a classroom where the teacher was not going to have the ability to teach them, would I want technology to step in? Most certainly, yes. But I'd much rather be in a classroom that's resourced with a teacher that supported the student's needs holistically more than just filling them with more letters.

Dan Schwartz (17:30):

So there's that second grade teacher you never really like, Jason. On the other hand, there's a computer that has Luke Skywalker praising you for getting all the right answers.

Jason Yeatman (17:40):

I think it has a place, but I think I'm still sticking with that we want to empower teachers, empower the compassionate, caring humans to be the lead here. And technology can be a great supplemental resource, right? And I think we often ask the question in the wrong way. We're often asking, "Can AI do this" or "can technology do this?" But we should be asking, "Do we want technology to do this? Do we want AI to step in here?" And I think often the answer, I don't know, when you think about your kids, do you want them just learning exclusively from the computer? No, you want it to be a supplement where for the individualized parts of it, sure that there's an engaging teacher that's working with students on their skills, practice is a great thing to do through an app. If you're just needing more practice to master skill, a teacher can't spend that much individual one-on-one time. Tutoring is a great option, but if financial resources preclude tutoring, great place for technology to support learning. But not to be the place where all the learning's done.

Dan Schwartz (18:40):

Okay.

Denise Pope (18:42):

I agree 100%. I love having this conversation with Dan in the room, Jason, just come with me every time I have this conversation with Dan. I love it.

Dan Schwartz (18:50):

Hey, just to be clear, I think teachers are really important. My point was simpler, which is our diagnoses may be more precise than the available actions to the teacher.

Jason Yeatman (19:00):

Yeah, that is a good question. And what we think about with ROAR, so ROAR has gone from a research tool to a tool working at the scale of systems, and what we like to think about with school districts is how to build an effective multi-tiered system of support. So that's the goal here is that you're catching problems early and you're finding students that just need a little bit more help. And then as some of those students that just need a little more help are getting the help they need, they're fine. And others continually need to be brought to a more intensive directed, sometimes one-on-one, and this is where there is this interplay where technology can thoughtfully fill a lot of these gaps. But it's not the solution on its own, right? It should be integrated with a thoughtful, multi-tiered system of support.

Denise Pope (19:49):

Okay, building on what you just said, I still want to know how you can tell the difference between a kid who's just having a little bit of a slower role to learn how to read, may need a little bit of one-on-one, versus someone who is going to be diagnosed, have an IEP, which is going to get them special support, can you tell that in kindergarten or is that something you have to give to older students?

Jason Yeatman (20:14):

Now that's something you look at over time, right? So back to this idea of a multi-tiered system of supports. First, everyone should have high quality basic instruction. And you should screen kids and some kids are going to be struggling, those need a little additional support. And then there are those that are going to keep struggling on the screen or struggling on all the assessments and yeah, those are the ones that are going to need a different level of support. Is it important to have a diagnosis? That's really a policy question, not a science question. Because what I would say is assessments provide data on who needs support and what. If you have a system that's resourced to provide students the support in the areas they need, the diagnosis itself is not important. However, for a policy standpoint, the diagnosis gives you access, sometimes, in certain scenarios, gives you access to services you wouldn't otherwise have.

Dan Schwartz (21:05):

Let me switch a little bit. So I taught English in middle school and I don't remember ever giving a straight up reading assessment.

Denise Pope (21:13):

I taught it in high school and I never gave it, ever.

Jason Yeatman (21:16):

Let me give you now the two areas, you guys just asked about the two areas that I'm most excited about with ROAR, and it is upper elementary school into middle school and high school. Let's take this Dan teaching middle school, and there's probably a bunch of students... So students come into your class, you probably don't have a lot of information about them, about their foundational skills, you just have students that are struggling across everything. I would be willing to bet that you take that section of your class that's just struggling with everything, third, half of those students, the issue is really that they never mastered foundational reading skills. They're still stumbling through decoding words. They're still struggling to read fluently enough to read large passages, to digest information, to read the instructions on a math test.

(22:01):

And this is what we find, this is through research partnerships with many dozens of middle schools and high schools, we're finding that of the students that are not meeting standards on the state tests, a huge portion of those are also struggling with the foundations of reading. Now, I find this both sad and concerning, but also a place of hope because it's actionable. We know how to teach reading. When we think about how do we teach the middle school student that is not meeting many of the standards, how do we teach all these standards? That's a very complex problem. But how do we teach them to decode words and read fluently? There's an extensive literature on how to do that.

(22:42):

The challenge though, for you, Dan, as the English teacher, I guess not really within the structure of your day, right? So this requires thinking about structuring middle school differently if we're going to start addressing these challenges. Early screening legislation, the ideal is that these challenges all get addressed early and this fact that I'm telling you is no longer a fact, but in the real world we live in, I think that there's a lot of power to thinking about addressing reading challenges at whatever age they are.

Denise Pope (23:14):

I think it's awesome that ROAR works not just for the young kids then, right? You can use it with an eighth grader, you can use it with a 10th grader, you can use it with an adult. The population in prisons and whatever. A lot of people I've heard in the adult correctional institutions are struggling readers as well. I mean, this is a problem that goes well beyond.

Jason Yeatman (23:34):

It's a huge problem. And this is why we've decided to keep ROAR at Stanford and run it through a research practice partnership model as opposed to spinning off a company. Because there's so many important research questions and also ways to serve underserved populations of society. You brought up one of prison populations, that's an area where assessments are not common, but as they do make it there, they could play a really important role.

(24:02):

The other thing we're working on now, hopefully when this podcast goes live it'll be right around the time of our launch of ROAR at Home, our parent portal. Our goal is to democratize access to high quality data to any parent that wants to learn more information about their kids' reading skills, their kids' math skills. We have math assessments in ROAR as well. We also, back to the example you gave at the beginning, are building a suite of vision assessments. There's a lot of aspects of way your visual system processes information that we can also measure through tasks we've developed, and this will give parents access to score reports in real time. It'll also allow parents and families to contribute to and participate in research.

Denise Pope (24:43):

Okay, I love that there's this idea of ROAR at Home and there's a part of me that's a little bit afraid because we have some really hyper, uber-

Dan Schwartz (24:53):

I can solve it. I can solve it, Denise.

Denise Pope (24:56):

Wait. But I didn't even say the problem yet.

Dan Schwartz (24:59):

Yes. The answer is, Jason, they're not allowed to use the assessment on their kid more than once every two months.

Denise Pope (25:05):

Okay.

Dan Schwartz (25:05):

Did I get it right, Denise?

Denise Pope (25:07):

Yeah. There's someone who's going to be like, "Do you read yet? Do you read yet? Let's do it again. Wait. Oh my gosh, now I need all these interventions. We're going to pay all... We're going to workbook you to death. We're going to flashcard you to death." That's my fear. What do you think?

Jason Yeatman (25:20):

I think that we are neither going to prevent nor cause that.

Denise Pope (25:25):

That's like you're going to push that off to policy and economics and all that. Someone else is going to solve that problem.

Jason Yeatman (25:32):

You know, if a parent wants to test their kid 1,000 times and buy every single intervention program, whether or not I'm involved. But no, I mean to Dan's point, we can set up guardrails here. And one of our goals is to provide high quality information, right? And we provide instructional recommendations, we don't provide recommendations for a specific curriculum or a specific product to buy. We provide information on approaches to work with students that have different kinds of challenges. So our hope is really to curate the scientific literature. We're always hoping to do better on this. We're a small team. Unfortunately, we don't have enough people to do as comprehensive a job as we'd like. That's the goal, is to curate the scientific literature in a way that a parent could digest and learn about their child's strengths and weaknesses. Not test them every week, maybe test them a couple of times a year. Quarterly, quarterly is pretty good. Can see growth curves then, you can chart growth over time. Monthly would be okay too.

Denise Pope (26:31):

Oh no, no, no, no, no. There's so many other things little kids need to be doing than sitting and testing. But I love the availability of this for people at home, for schools that aren't using it yet or whatever. We want early intervention for sure. Jason, thank you so much for being here. Some final words?

Jason Yeatman (26:51):

For me as a laboratory scientist, as Dan opened with, I'm a neuroscientist, my career started off really examining details of brain development, intersection between brain development and learning, and I find that being in the real world, doing research in the real world, engaging with the world just makes the core research so much richer as well. We discover things we would not discover living in the confines of the lab by working with diverse populations from around the country, from different school districts across different states.

Denise Pope (27:23):

It's such a huge point, and I love that you kept it at Stanford instead of spinning it out to a company, which a lot of research people do, so that you can continue to improve, so that you can continue to make it better, so that you can continue to study it. So that is just kudos to you, Jason, and your team. Super, super excited.

(27:42):

Dan Schwartz, looks like you were going to say something. Final words here.

Dan Schwartz (27:45):

So summarizing, I think this is an amazing demonstration of what science and smart design can do.

Denise Pope (27:53):

Yeah.

Dan Schwartz (27:54):

Right? Where you really take what we know about the world and render it in a form that's just so helpful, so useful. It's not guessing all over the place and really, it's very precise. I think this is a testimony to what educational research can do.

Denise Pope (28:10):

Thank you, Jason, for being here, and thank all of you for joining this episode of School's In. Be sure to subscribe to the show on Spotify, Apple Podcasts, or wherever you tune in. I'm Denise Pope.

Dan Schwartz (28:21):

And I'm D-AN.