Getting down to facts: Education policy at scale

Susanna Loeb (00:00):

We actually ended up with 55 studies in this project, but it came from, first, that listening tour and then really thinking together about how we could best address all the issues that came up.

Denise Pope (00:16):

Welcome to School's In, your go-to podcast for cutting edge insights in learning. From early education to lifelong development, we dive into trends, innovations, and challenges facing learners of all ages. I'm Denise Pope, senior lecturer at Stanford's Graduate School of Education and co-founder of Challenge Success.

Dan Schwartz (00:39):

And I'm Dan Schwartz. I'm the dean of the Graduate School of Education and the faculty director of the Stanford Accelerator for Learning.

Denise Pope (00:49):

Together, we bring you expert perspectives and conversations to help you stay curious, inspired, and informed. Hi, Dan.

Dan Schwartz (01:00):

Well, hello, Denise. I got a question for you that's relevant to today. So there's a new governor who's coming into a state, and states have a lot of responsibility for education, more than the federal government, and you have to write a brief to update the governor on the state of education because they don't know. They need to learn this. So how are you going to choose the topics? This is a huge space. Are you going to do something from the parent-teachers association or are you going to talk about pre-K curriculum? How do you choose what to tell the governor?

Denise Pope (01:33):

Oh my. Well, do I only get one topic? I mean, there's a ton. How many pages could the brief be? How brief should the brief be?

Dan Schwartz (01:42):

Somewhere between one and a ton, I think.

Denise Pope (01:46):

Dan, where do you even start? If it was California, we have so many issues. We've got money issues. I think we're the state with probably the most multilingual learners. We have absenteeism through the roof. We're still kind of reeling from COVID and all of the COVID stuff that happened. We have technology. We have, you know, mental health. I don't know.

Dan Schwartz (02:07):

For me, I would look at my kid and say, "My kid's really good at this. Why don't we have more classes?"

Denise Pope (02:13):

Your own kid?

Dan Schwartz (02:14):

Yeah, sure.

Denise Pope (02:14):

Dan, no way. Uh-uh.

Dan Schwartz (02:17):

It's-

Denise Pope (02:17):

I'm calling you on this.

Dan Schwartz (02:17):

Denise, it's all about me. It's all about me and my family.

Denise Pope (02:20):

Okay. This is not useful, but I am sure that we have someone coming on today who has a process better than just looking at their own kid.

Dan Schwartz (02:28):

So our guest today is the fabulous Susanna Loeb. So she has to write this brief for the governor. It's called Getting Down to Facts, and this is the third governor, so it's the third report they've done, so we're going to find out how she does this. Susanna's a professor at the Graduate School of Education. She studies educational policy and its role in improving educational opportunity for students, among many other things. So she left Stanford for a while, in a great mistake, she went to Brown University and was the director of their Annenberg Institute. And there, she started the National Student Support Accelerator, which aims to expand access to high-impact tutoring with strong relationships. But she came back, she's doing even more and more and more. And so, we are really lucky to have her here with us on the podcast and at Stanford. So welcome, Susanna.

Denise Pope (03:19):

Yay. Yay. Welcome back. I'm excited to talk about this new report. So Dan said she was made to write this report when the new governor came in way back when. How did that even start? I know it's called Getting Down to Facts, so maybe give us a little bit of a background.

Susanna Loeb (03:36):

Sure. So Getting Down to Facts started in the Schwarzenegger administration, and it really was aimed at stepping back and taking a look holistically at the school system. It is really easy to say, "I want this thing changed," or "I want that thing changed." But it takes a little effort to step back and say, "Okay, is the whole system functioning the way that we want it?" And so, we brought a bunch of researchers together at that time to provide evidence on where California's system was really set up to be effective and where it wasn't, and there were some really clear problems at that time. And so, we suggested some things there. And then, we did that again before Newsom came in to office in 2018, and now we're doing it again for this next governor.

(04:25):

The reason it's called facts is because there is so much kind of emotion involved in education and making sure that the system is really set up for the kids that we know really well, but the facts are very helpful as a basis. Even with the same facts, we can disagree on what the right direction is to take, but at least if we're on the same page with what's actually going on, it can help us be more productive in our discussions about how to best change the system so that it can function in the way that we want it to.

Dan Schwartz (04:58):

So what's an example of something that came out of Getting Down to Facts?

Susanna Loeb (05:01):

You know, the reason that I did have been working on Getting Down to Facts 3 is because it really does look like Getting Down to Facts 1 and 2 helped get California to a much better place than it was before. The first one, it was a little bit easy to see what was wrong when we went in there because we had this really illogical finance system. You could have a district that was very much the same and getting really very different amounts of money from the state. And then, you could have districts that had much greater needs and costs and they weren't getting any more than districts who weren't. So what we did in Getting Down to Facts I is really bring to light the inequalities and the illogical nature of that system, and it led to, at least in part, to the local control finance formula that we have today, which is at least a very systematic approach to funding schools.

(05:54):

We can debate on exactly how much goes to what kinds of schools, but at least we know that there's a logic behind it. And so, for Getting Down to Facts 1, I think the biggest thing was this local controlled finance formula and maybe also reducing some of the requirements on districts. We had this huge education code, and we provided evidence that there were so much there that the districts didn't even really know what was there, and so we got that cut down a bit. And then, Getting Down to Facts 2, one of the things that came out of that, which I think was really helpful, was the issues we were having in early childhood education, not so much before students in the zero to three range, but more in preschool and as a result, this, I think, really led both to universal transitional kindergarten, TK, and also to just a new focus on the skills that educators need and the curriculum and materials they need to address early childhood in TK through 12 schooling.

Denise Pope (06:55):

Susanna, huge wins. I mean, you changed the way schools are funded in an entire state, and our state, as people know, is huge. So amazing. Kudos to you.

Dan Schwartz (07:06):

So back to the Getting Down to Facts, you go and you say, "Bob, go get me some facts." Like, how do you do this?

Denise Pope (07:15):

Yes, Bob.

Susanna Loeb (07:16):

Yeah, exactly. Well, somewhat similar to that. So we start actually with a listening tour around California. So there, again, a billion different things you could look at in education, and so we wanted to go in with a research agenda that really reflected the issues that Californians recognized that we had and the goals that they had. So we started by talking to a whole range of policymakers, advocacy groups, families, to get a sense of where the interest was. From there, I put together a research agenda to answer some of those questions and went out and found researchers who were experts in each of those areas. And then, we came together a little over a year ago and looked at that research agenda together and identified pieces that were missing and also how there could be synergies across projects. As a result, we brought in some more researchers to do this. So we actually ended up with 55 studies in this project, but it came from, first, that listening tour and then really thinking together about how we could best address all the issues that came up.

Dan Schwartz (08:29):

If I went on a listening tour, I think I may only hear the things I care about. How do you-

Denise Pope (08:34):

Or just your son.

Dan Schwartz (08:38):

Yeah, exactly. Exactly. Until he disagrees with me. No, is it more formal than that or is it really, "No, we've heard four people say this and they've all said it in their different stakeholders and they've all pointed to the same thing"? Is that sort of how it goes?

Susanna Loeb (08:54):

It was a lot of that. We went into it in many ways humbly because while I am a good researcher in many ways, I don't know this state the way the people we were talking to know the state of California. We really did seek out these different perspectives because they're essential for making this a worthwhile project. So luckily, being self-serving to the project, you know, I want a project that in the end is really useful, meant that it made sense to be super open to what all the people we talked to were interested in.

Denise Pope (09:30):

Really amazing. And 55 different research projects, now, you said that was a year ago. Some of these obviously had been started and were ongoing before them because you can't just do all of that in a year, right?

Susanna Loeb (09:42):

Well, the way that we design the research isn't that the researcher is supposed to be able to do a study that tells us all the answers, but that the researcher is supposed to go and see what studies were out there that could speak to this and then what they could add on over the course of a year that would make that even better for answering those questions. So it's a combination of those two. It is relatively fast turnaround. I know we, researchers, are slow, but we did make use, of course, of the research that was there, adding on new data, doing more interviews, those kinds of things. We did a lot of interviews with different school leaders, principals, superintendents, other district leaders, as well as interviews with families around the state. So we added on using new data collection as well as administrative data that's already there.

Dan Schwartz (10:35):

Oh, that's great. So now, you've got these 55 research proposals. You can't go to the governor with 55 things. And so, you get these 55 researchers in a room and have them duke it out? What happens next?

Susanna Loeb (10:50):

Yeah, pretty much. So we came together again, and the idea was to take these 55 studies that each were designed in a way that they collected some information, but clearly not all information about a topic we might be interested in like the teacher workforce or multilingual learners or the finance system. And so, we got together in groups around topics and decided what the key takeaways were, and in each of actually 22 topics, we are having a summary of what all the papers that speak to those say. And so, we'll have 22 topic area policy and research briefs that if, let's say, the new governor or the legislature wants to go into a particular topic, there'll be a brief that summarizes what the reports say, and we'll also point towards those reports. But then, in addition to those 22 briefs, we will have one summary paper, which it pulls together what's there. Of course, it doesn't give all the richness, but it really highlights some of the main takeaways of the project.

Dan Schwartz (11:56):

So are we allowed to ask what some of the main takeaways are?

Susanna Loeb (12:01):

Yes, you are.

Dan Schwartz (12:02):

Better line up with my kids' needs. I'm telling you.

Denise Pope (12:04):

Yeah, exactly.

Susanna Loeb (12:05):

They definitely do. So I think actually on a positive note, one of the key takeaways is that California really has made some progress over time. So we've seen a much more logical school finance system. There's been huge investment. We used to be really very low relative to the rest of the country on how much money we were giving to schools. That's increased a lot. We've also invested in transitional kindergarten, in TK. We have a lot of individual projects that have worked well. So there are small scale early literacy programs that have worked. The community schools appears to have benefits for students, so we have a stronger base than we did before.

(12:56):

But what really came out of this in terms of the next step is that California, while it does these individual things quite well, has not managed to have a system that can consistently improve instructional quality at scale. So some other states are able to come in and say, "We would really like our math to instruction to improve in the state." They are very targeted in the kind of math instruction that they envision and then they put in professional development supports and curricular supports around that vision. California just hasn't done that.

Dan Schwartz (13:41):

So I've had some interactions with school districts and they're all using different math textbooks, but in these other states, are they saying this is the curriculum you have to use in your district?

Susanna Loeb (13:51):

Yeah. So generally, at least they give them a small number to choose from, and it is possible to give waivers, but that's a really good example. So we have these ABCs of things that California is not doing that they really could do, and A is alignment and accountability, so we're kind of all over the place. Nobody knows who they are accountable for, and we just don't have that kind of system in the state that would really let us know where the issues are and respond to those issues. The second B is burden, and that's what you're talking about. So California puts a lot of burden on districts. Instead of incentivizing them to do well, they'll instead give them freedom, except that they'll ask them to do a lot of things like bring people together, write these long plans, and no one ever reads those long plans, but they are there and they take a lot of effort.

(14:49):

And the curriculum is a really good example of this. So California spent a long time reviewing 60 different math curriculum just to see if they get above a bar. And so, the state says all of these curriculum are good, but each of the a thousand districts in California, for them to spend time reviewing all of those different curriculum is a huge amount of time and largely a waste of time. If California would just limit the number and then provide stronger supports around capacity development for those curriculum, we'd be in a much better place. And so, along with the A, which is alignment and accountability, the B of burden, the C is capacity building. And it's really hard to build capacity in this kind of approach that we've taken so far.

Dan Schwartz (15:37):

And that's because there's so much variability that you're permitting with all these different approaches. It's hard to say, what's the capacity that we're focusing on?

Susanna Loeb (15:47):

That's right, and it's hard to provide capacity building supports in every single thing that a district could possibly do. If we were more focused on what goals we want and what kinds of instructional approaches and educational experiences we want for students, we, as a state, would be much better at helping districts provide that kind of instruction and those kind of experiences for students.

Denise Pope (16:14):

I mean, it's such a tricky thing. I mean, we are a huge state. We're so diverse. We have tiny, tiny little one school districts and then huge districts with so many schools. And I mean, I know just the math wars alone over the math curriculum could just knock your socks off, right? And that's only one little tiny, tiny thing that a district's supposed to do, and you've got one superintendent and a school board. And guess what? Anyone can be on a school board. You don't have to have any kind of background in education. So this is a huge task.

Dan Schwartz (16:47):

So Susanna, I'm a school district, and because of your report, the governor decides that there's going to be four possible textbooks for math. I'm the superintendent and I've got, "Hey, I got my own ideas. I'm brilliant, and I want to get this done, and Susanna's just lobbying for her point of view." How do you defend yourself from the idea that you're just lobbyists for your point of view?

Susanna Loeb (17:16):

Yeah, that's really a great question. I think that's why we named the project Getting Down to Facts and why we really are quite careful about providing the facts of what we found. So as an example, we did a lot of interviews with district leaders, and they talked about the time cost and the burden of having to make so many choices. What we present is just that, that districts are struggling with this, that they're spending a lot of time on it. We actually don't take it to the next step and say, "As a result, you should do exactly this thing." We're really trying to say, "There is a lot of burden from this. There's a lot of burden from some of the requirements of the Local Control Funding Formula right now, and the requirements on that and then the additional requirements put on by special ed."

(18:13):

So it's very much saying, "This is what is." We do have in the reports, we refer to what other states have done that might be different and might provide examples, but we are really careful not to take that next step and say, "This is what you should do."

Dan Schwartz (18:32):

Oh, that's really interesting. So you're not offering solutions. You're sort of saying, "Here's the space we need a solution for. And then, there are people who've worked on this, you might look at this to see how they've done it."

Susanna Loeb (18:44):

Exactly.

Dan Schwartz (18:45):

Yeah. No, that's really smart.

Denise Pope (18:47):

So what happens? You write this report, you put it on the next person's desk. What happens then? Are you still in touch with this governor? It's not just the governor who looks at this report, I'm sure.

Susanna Loeb (19:00):

Yeah. So the first thing we do is try to get this information out, so we have a conference in Sacramento, we also do one, that's the biggest one, and then there are ones in other parts of the state that really hopefully get people to discuss- learn about and discuss the findings. And then, we have videos that we've made of each of the studies, so we try to make them really quite accessible to a broad range of people. And then, we hope that as we get this information out there and accessible to all the advocacy groups that are in the state, as well as to the decision makers themselves, these reports can be useful.

(19:45):

And each of us, the 70 or 90 people involved in this project are willing then to go as individuals to participate in meetings with decision-makers, working with the advocacy groups, creating sample policies. So there are things that go on after the project is done, but because this is very much Getting Down to Facts, we're really focused on getting those facts out and the discussions going. After that, we're no longer really part of the Getting Down to Facts project, and each of us can go and try to work on the topics that we care most about.

Dan Schwartz (20:22):

So Denise, I'm a dean. I have lots of faculty. The idea that you would get 60 or 70 faculty on the same page-

Denise Pope (20:32):

I know.

Dan Schwartz (20:32):

... going out on the road to explain the importance of this is stunning to me. I mean-

Denise Pope (20:37):

No, I know. And I've seen some of the topics.

Dan Schwartz (20:38):

... I love the faculty. Excuse me. I love the faculty, but they are independent thinkers, so this is pretty amazing. So in-

Denise Pope (20:45):

Well, and I've seen some of the topics. They vary widely, right? So it's not just getting people to agree within their own specialty. It's also, then you got to get everybody to look at, "Okay, we're moving from 55 to 22 topics to, and what are the ones that we're going to really hit hard?" I mean, it's a lot.

Dan Schwartz (21:04):

This is pretty amazing getting people together like this.

Susanna Loeb (21:07):

I'm very thankful. The researchers have been such great sports about this. They've really brought their A game to the project and all of their hearts, so it's been a really nice thing to see.

Dan Schwartz (21:20):

So here's a different... This is incredibly informative, but a lot of policy research is about what has been done. And so, how do you handle the disruptions that are going on, say, with AI, the possibility of tutoring? How does the report anticipate innovations?

Susanna Loeb (21:39):

Yeah, that's a great question. I actually think the focus of the reports on creating a system that can respond to change and improve instruction is particularly important in a time like this when we have so much change going on. We have change in the role that the federal government's taking. We have changed in our understanding of learning and in brain science, and we have definitely the impact of AI in schools and on our workforce. In this time of change, it's really good to have a well-functioning system, but part of the report is on what do we want now that we know this? How can we make, particularly high schools that are disengaging for many students, how can we make those rich experiences that really speak to students and help them develop the skills that they'll need to thrive? How do we leverage AI?

(22:34):

So what we have in this group of studies is not only a look at the kind of governing systems in California, the capacity in California, but where could the state go? What are the opportunities with AI? How could those new tools help us to overcome some of the barriers to getting students the individualized attention that they need, the creative experiences that are engaging for them? So we try to balance these kind of look into the future aspirations, what is possible with how do we get a system that will allow us to respond to these possibilities.

Denise Pope (23:17):

So important. So you narrowed the 55 down to 22 or something, right? Just like a quick tick off of what some of those topics are, and I'm thinking if I'm a parent or an educator, what should I know?

Susanna Loeb (23:29):

Yeah. So we have lots of topics. For example, we'll have topics on special education and multilingual learners on the goals families have for schools on what's happened in the finance and in achievement, how California looks in those areas. The workforce one is really interesting because we have made some progress in the educator workforce in California in terms of increasing the number of teachers that are available. That said, we have very little supports for teachers once they're in the classroom. So if, for example, we want a new math curriculum or we want to respond to the changes that AI is bringing in, we don't have really good ways of reaching teachers and helping them with those changes. So there's some issues there that could be a good place to look as well as we have most of our teachers have to come in through what's called a fifth year where we don't have undergraduates going into teaching.

(24:35):

There were reasons that we did that because we thought that it would be helpful to have an undergraduate degree before getting your expertise in teaching. But as it turns out, you lose a lot of people because they need that extra year, and you also don't have that full four years to develop some of the skills you need to be a good teacher. So there are things like that in the workforce that are worth looking at. There's so many different topics like this once you get into the details, but the way that we've structured it where we've got these summaries in each of the areas hopefully will be useful as opportunities open up in specific domains that you can go in and see what we know about those domains.

Denise Pope (25:14):

Well, it's kind of nice if you're a parent or an educator with a specific interest, it sounds like you'll be able to go and kind of find that and dig in and learn more about that, whether that's absenteeism or mental health or AI or teacher workforce. I'm just curious to know what's going to come out. I wish I had a crystal ball because you looked at Getting to the Facts I, and we saw kind of a coalition around getting the finance changes, among other things, I'm sure. With number two, sort of the biggie was TK and continuing with the finance, so I'm just so curious what number three is going to bring. If we can get more systematized professional development for teachers, I would bow down to you, Susanna. I mean, every time I go to a district, they're like, "Yeah, we have a half day, and we have to do CPR, and I don't think we're going to be able to train our teachers to do the newest thing." And you're just like, "Ah."

Susanna Loeb (26:05):

Yes, you and me both. I'd really love to have a crystal ball, but I also really hope that we get the California education system to be a model for the country and the world about the kinds of rich experiences that we can provide to students and how well they're doing and how much they're thriving.

Dan Schwartz (26:24):

Speaking of being a model, so Getting Down to Facts, other states look at it like what you've done?

Susanna Loeb (26:29):

I've had people reach out about the process and about using the process that we're using for California and other states, so I think that's one. And then, there's certainly, within the 55 studies, studies that speak well beyond California, and so those are useful. We try to get those out, published in different places, different kinds of journals and newspapers so that not only California, but the rest of the country can make use of them.

Denise Pope (27:00):

Dan, I'm always putting you on the spot at the end of the show. What did you learn? What was your aha?

Dan Schwartz (27:06):

There's a lot of content that I didn't know like the finance reform and things like that. But I'm taken with the process. It's simple, but it's hard. You get research professors, professors who do research to go find facts, to figure out what exactly is going on in a system that's so complicated, that it's really hard for anybody to feel the whole of the elephant, so to speak. It's such a great use of talent. And then, the distillation process where you actually get it to people who now know something so they can start to make decisions based on at least some evidence, it's a brilliant thing. I sort of want five faculty to come in and look at my personal life and get the facts and then give me a report and sort of say, "Dan, alignment is a problem you have at home." So I don't know.

Denise Pope (27:56):

Capacity, also a problem.

Dan Schwartz (27:59):

Yeah.

Denise Pope (27:59):

No, I agree. I think this is such an amazing gift. If I was the new governor, I'd be so thankful for this, but also, I just think it's an amazing resource that you're able to do and pull all of these disparate things together in a way that's really understandable. So it's a huge gift. It's a huge resource. Susanna, we're so lucky to have you, and I think everyone learned a lot, and we're really excited to see. We'll look in that crystal ball and have you back on. So thank you so much for coming and for joining us, and thank all of you listeners for joining this episode of School's In. Be sure to subscribe to the show on Spotify, Apple Podcast, or wherever you tune in. I'm Denise Pope.

Dan Schwartz (28:39):

I'm Dan Schwartz.