Learning design: AI and machine learning for the adult learner

Candace Thille (00:00):

Where is the learner now? Where are they trying to get to? And what kind of strategy is going to be the most helpful?

Denise Pope (00:09):

Welcome to School's In, your go-to podcast for cutting-edge insights in learning. From early education to lifelong development, we dive into trends, innovations, and challenges facing learners of all ages.

(00:25):

I'm Denise Pope, senior lecturer at Stanford's Graduate School of Education and co-founder of Challenge Success.

Dan Schwartz (00:33):

And I'm Dan Schwartz. I'm the Dean of the Graduate School of Education and the faculty director of the Stanford Accelerator for Learning.

Denise Pope (00:42):

Together, we bring you expert perspectives and conversations to help you stay curious, inspired, and informed.

(00:52):

Hi, Dan.

Dan Schwartz (00:53):

Denise, it's great to see you. Great to hear you.

Denise Pope (00:56):

We have a fun show today. I'm going to open us up with a question for you. This might be hard to answer. I don't know. I can't decide. A lot of people look to technology as a way to really change – a super game-changing way to change – teaching and learning. And so as you think about technology, and I don't mean a chalkboard or pencil or whatever – I know some people consider those technology – but as you think about technology, what do you think is an example of the best technology that's made the biggest difference?

Dan Schwartz (01:31):

Oh, that's easy.

Denise Pope (01:32):

It's easy? That's an easy question? 

Dan Schwartz (01:33):

That's easy. It's easy. If we ignore hardware, the number one technology for learning is YouTube.

Denise Pope (01:39):

YouTube?

Dan Schwartz (01:40):

By far.

Denise Pope (01:41):

YouTube?

Dan Schwartz (01:42):

Yeah. How many . . . by far. You use it to fix the toilet. That, if you're a teacher, you show a YouTube video of the planets orbiting because it's so much more effective than looking at a picture in a textbook.

Denise Pope (01:54):

Interesting. First of all, I did not think you were going to say YouTube. And second of all, I don't use YouTube in my classroom, is that bad? I use YouTube all the time. I do. I use it when I need to fix something. I use it, well, mostly when I want to fix something, actually, yeah?

Dan Schwartz (02:12):

Yeah, that's a good use for it. If it's some sort of procedural thing, YouTube's pretty good. But you're probably . . .  If you're trying to teach science, YouTube's got a lot of good videos, things like that.

Denise Pope (02:22):

I can see that. Interesting.

Dan Schwartz (02:22):

But I don't know. I don't know.

Denise Pope (02:27):

Well, this is why we're so lucky. We have an expert today who can talk about probably a lot more than YouTube but can certainly weigh in to that question. I'll let you do a little introduction, Dan.

Dan Schwartz (02:38):

Well, thank you. Thank you. This is a YouTube video of me introducing Candace. Hi, it's my pleasure to introduce Candace Thille, who is a professor at the Graduate School of Education, and she particularly focuses on adult learning, say, in the workplace, but also community college, and she's a big-time commitment to technology.

(03:02):

So Candace, is there a technology that's done something in education? Like, is there a big winner there?

Candace Thille (03:10):

You know, I actually wouldn't have picked YouTube as my number one

Denisd Pope:

OK?

Candace Thille:

But I figured I'd get a chance to weigh in on that. And I agree with you both that YouTube is great for –  I mean, I use it when I want to learn how to pan fry salmon. I pull up a YouTube video. But what happens is the video is telling me what to do, and I'm doing it while the video is showing me, and I can stop it and rewind it and go, "Oh wait. How long am I supposed to put it on for? How much salt should I put on?"

(03:38):

And so I would say that the reason I learned how to cook salmon . . . the first time, I do it that way, step by step. The second time, I might use YouTube, have it playing, but I'm kind of doing it, and then it's like, "Oh wait, yeah, there's that part." And then the third time, I might quickly just review it and then just go cook the salmon. But, so you could say that the YouTube taught me how to cook the salmon.

(04:04):

I would say that the YouTube was a resource I used while I was learning how to cook the salmon. And what was really teaching me how to cook the salmon was I was cooking the salmon, and I was getting feedback from my experience that oh, I turned the heat up too high, I put in too much salt, it tasted funny, and I kept modifying what I was doing. So I was building my knowledge through my experience of what I was doing, and the YouTube was helping me perform that experience.

Dan Schwartz (04:34):

Oh, so your conclusion about the most important technologies will be ones that can give you feedback?

Candace Thille (04:40):

Well, feedback is important.

Denise Pope (04:43):

She also said experience is the best teacher.

Dan Schwartz (04:47):

I'm saying the key to experience, besides feeling the heat of the pan in your hand, is saying –

Denise Pope (04:52):

Don't underplay that.

Dan Schwartz (04:53):

No, no, I know that's very important.

Candace Thille (04:54):

And the taste of the salmon.

Denise Pope (04:56):

Don't underplay experience.

Dan Schwartz (04:58):

Well, the taste of the salmon is feedback.

Candace Thille (05:00):

That is. It is feedback. So feedback can come in many forms. It can be someone directly saying to me, "You have the stove up too high," or it could be, ooo, my salmon got cooked more than I wanted it to, maybe it's because I had the stove up too high. They're both feedback.

Dan Schwartz (05:18):

Right. And so, my proposal was that the technology that you are going to like the most is one that can provide the learner with feedback as they try and accomplish a task.

Candace Thille (05:28):

That's part of it, but not just any kind of feedback. Feedback that is . . . well, there's that old saying. Finish this sentence: Practice makes . . . 

Denise Pope (05:38):

Perfect.

Candace Thille (05:40):

Nah, practice makes permanent. Practice with feedback. And I would add and the opportunity to refine my performance based on that feedback, makes perfect.

Denise Pope (05:52):

Wait, let's unpack that. Practice, and what does that practice look like, Candace? Because I know there's different kinds of practice. Right?

Candace Thille (05:59):

I would say deliberate practice.

Denise Pope (06:02):

And for the listeners, say what that is because I know that's a particular kind of practice, deliberate practice.

Candace Thille (06:08):

So, if I'm trying to learn how to play tennis, for example, I could just play a game. That’s what a lot of us do with our friends – go out and play a game – and I'll learn how to play tennis. However, if I get a coach who I'm intentionally trying to improve my tennis game, they'll look at me and say, "Oh, your problem is your serve, and it's that you're not throwing up the ball high enough, so I want you to practice throwing the ball up and getting it to just the right height."

(06:34):

Now, that's not as much fun as playing the game, but it will – if I get that component of my game –  improve it. Then I can incorporate that into my game, and I'll play better tennis.

Denise Pope (06:46):

So deliberate practice is when you're practicing, but you have somebody telling you specifically what you should focus on. Yes?

Candace Thille (06:52):

It is practicing in a way where you are deliberately trying to improve or learn or achieve something.

Denise Pope (06:59):

So it's experience with deliberate practice, and then you had something else.

Candace Thille (07:04):

Yeah. So, part of the deliberate practice is you try something. You get feedback on it. You have the opportunity, then, to use that feedback to refine your practice.

Dan Schwartz (07:16):

So back to my assertion. For you, the technologies that are going to make the biggest impact are the ones that can provide feedback? That they're interactive?

Candace Thille (07:25):

That they're interactive, and they provide feedback that is targeted and timely.

Dan Schwartz (07:31):

Targeted and timely. I like that. So YouTube does not do that?

Candace Thille (07:34):

YouTube does not do that.

Dan Schwartz (07:34):

Not yet.

Candace Thille (07:35):

Not yet.

Denise Pope (07:38):

So . . . Dan's wrong. I like that. I like it when Dan's wrong.

Dan Schwartz (07:41):

Nice! Nice summary statement.

Candace Thille (07:45):

OK, we've learned everything we needed to learn here.

Denise Pope (07:47):

Exactly.

Dan Schwartz (07:47):

End the show.

Denise Pope (07:49):

But Candace, you really do work with adult learners, and you have this vast experience doing that in lots and lots of different ways.

(07:57):

So when you think about that versus maybe a younger learner or a child learner, what would you say are some of the key differences?

Candace Thille (08:06):

Oh. One is physiologically they're different in terms of brain development. Secondly, and I think actually probably the bigger difference, is experience and prior knowledge. Adults just have more of it. So much experience, so much new knowledge, is built on prior knowledge.

Denise Pope (08:25):

So let's talk about experience, though, because I also know – just like I said practice makes perfect – there's also: It's harder to teach an old dog new tricks. So how do you factor that in? 

Candace Thille (08:40):

Yeah. So, that adage oversimplifies the situation. If the new trick that you're teaching me is lined up well with my prior knowledge, then it's actually easier for me to learn that new trick. If it interferes with something that – the meaning that I've already made in the world, then it's going to be harder to learn it because I don't have a . . . As I said, all prior knowledge either supports or inhibits the development of new knowledge.

Dan Schwartz (09:06):

So I have a paper that the title is “Prior knowledge: you can't live with it, you can't live without it.” 

Denise Pope:

Oooo.

Dan Schwartz:

You like that?

Denise Pope (09:14):

I do. Wait, explain that. You can't live with it because . . .

Dan Schwartz (09:18):

Because it gets in the way. You have all these prior beliefs that can get in the way. You can be stubborn. But if you don't have any prior knowledge, it's so hard to make sense of things in the world.

Denise Pope (09:29):

Can't live with it, can't live without it. It makes sense. It makes sense, folks. I like it. I'm learning so much.

Dan Schwartz (09:35):

Candace, I've enjoyed this digression. It's been very good. Bring it back to technology. So I was setting you up to explain the kinds of technologies that you like to create, which are:  bring copious feedback, opportunities to practice, so they're kind of create a world –

Denise Pope (09:53):

For adults. Just to clarify, Candace, you specialize in doing that for adults?

Candace Thille (09:56):

For adults.

Denise Pope (09:57):

Yes.

Candace Thille (09:59):

So, the kinds of strategies that you need to help adults build sort of new skills in their field.  First off, you have to kind of know, have a clear articulation of what are the skills and knowledge that the adult's trying to build. So you need a clear target.

(10:14):

Then you need to kind of get a sense of where is the adult relative to that target. And once you have a clear idea of the learner's sort of current state relative to that desired state, then you can select a strategy that considers the kind of knowledge or skill that's being learned and things about the context and other things about the adult. And figuring that out is really complex.

(10:41):

So, if every time you had to make a decision about what should I do next, you need an insight into, where is the adult, where is the learner now? Where are they trying to get to? And what kind of strategy is going to be the most helpful? And you have to consider all of those features of the learner, features about the context, features about the thing that's being learned. That's too complex for most of our cognition.

(11:05):

And that's where the technology can really help. It can help on multiple levels of that very complex problem.

Denise Pope (11:13):

OK. It's so interesting because I teach curriculum construction, and I teach backward design, which is exactly that. Like if you're the teacher, you have to look at where you're going, look at where the learners are compared to where you want, and then what are you going to do to get them there. And so you're saying it's so complex, and I think teachers are doing this all over the world. This is what teachers are doing. So you've created technology to make that easier?

Candace Thille (11:36):

Yes. In the sense of,  if you think about it, there are several human actors in that learning system, right? There are the learners, and they're trying to make good decisions to help move them from where they are to where they're getting to. There's the teacher who's trying to support those learners to go from where they are to where they're trying to get to. Then in the adult learning world, there's often a third party, which is the designer that people are often teaching curriculum or using assets that they didn't create. And the designer's trying to figure out what kind of asset's going to help my target learner move from there to there. So everybody's trying to make good decisions to help move that learner forward.

(12:19):

But in order to make that good decision, having that insight about: Where is the learner? Where are they getting to? Given what we know about the learner, what kind of activity is going to best help them? That's a very complex decision that has to get made very quickly. Now, teachers . . . You'd say, the teachers, really good teachers, use their intuition, and I would say their intuition is kind of based on, it might be on their formal training if they take classes from you.

Denise Pope (12:47):

I think teachers learn from experience.

Candace Thille (12:48):

They totally do. They totally do. And their experience is their observation, right? Of either what worked for them when they were trying to learn it, or, when they've tried to teach people it, what seems to work. They look at their students and go, "That worked. That got it," and they file that away. And they build up an intuition based on their observations and based on their experience, right?

Denise Pope (13:09):

Right.

Candace Thille (13:11):

So, if you think about thousands and thousands of teachers building up those observations, and they don't really . . . there's no way that I get to benefit from all of your observations, so I'm making my decisions based on my limited set of observations. If we had a way where teachers could make those decisions, and we collected the output from the student learning of that decision on a moment-by-moment basis, then we could start to build a database of interventions and outcomes and include in that data set features about the learner, features about the thing being learned, and features about the context. And then we could start to use the AI to help discover patterns.

Dan Schwartz (13:59):

So you're creating teaching machine that can engage in deliberate practice?

Candace Thille (14:06):

Yes. The machine itself would learn, yes. But the fun part about building these sorts of things is . . . so the machine is learning and can refine its insight and decision-making, but how do you communicate that insight to the different human actors in the system so that they make their own decisions? It's important in these systems – even when the machine has insight to make a good decision – that the human actor, whether that's the teacher or the learner, always be in the position of making the decision. But helping support their decision with what the machine knows. You always want them to have agency, to be the one who is taking the action.

Dan Schwartz (14:53):

So Candace, I'm trying to think of concrete instances to wrap my head around this.

Candace Thille (14:58):

Okay. So, let's say I'm in the workplace, and I'm trying to develop a skill or some new knowledge. So using a system that I may have developed at some point, maybe I'm trying to learn the skill of how to give good feedback or good coaching to a direct report. So, there could be a system that would tell me that larger skill – giving effective feedback – and that might have certain sub-skills that are required for me to be giving effective feedback.

(15:28):

With me so far?

Denise Pope (15:29):

Totally with you.

Candace Thille (15:30):

You have a system that'd say, "I'm going to learn today about how to give effective feedback." And so then I'd start engaging with an activity, like possibly giving a virtual entity feedback and engaging that conversation. Then I'd get feedback from the learning system on it, and then I could try again, et cetera.

Dan Schwartz (15:49):

So there's some scenario where I am looking at a fake employee representation, maybe it's just text, and something's happened, and then I am giving feedback. Something like that?

Candace Thille (16:01):

Well, actually, three things happen. Let's say I'm in some kind of environment where an employee comes in. I give them some feedback. Then based on the feedback, they respond in some way. Well, actually, the system knows, given that feedback, then this is how an employee would respond to that feedback. So, I get the next step in the scenario.

(16:23):

The action that I took is coded in some way and gets sent to a database, essentially, that is tracking my actions that are associated with different skills. And, at the same time, that action is put into a model that then estimates how well or not I am developing whatever the target skill is. And so then I have a dashboard or an indicator that I can always look at that tells me, on this set of five skills I'm trying to build, where's the system estimating I am relative to where I'm trying to get to.

(17:02):

And then I can choose – this is where the agency part comes in – these are the skills I'm trying to develop. I see that I'm doing great on listening well. I'm not doing so well on giving a positive statement before I give a negative statement or whatever. And so I think I want to work on that more. So, then I can choose that I want to work on that. And then the system will say, "Well, here's some more activities that will, given what we know about you, will help you develop that skill."

Dan Schwartz (17:30):

Is that where the choice is? The agency is me choosing what to work on?

Candace Thille (17:35):

Me choosing what to work on, when I feel like working on it. Also, I can say that the system can be doing these estimates saying, "You're there. You're great," or it can say, "We don't think you're there yet." I can say, "You know what? I think I'm there, so I don't want to work on this anymore because I get what you're saying. I believe I understand it well enough."

(17:56):

So, agency on so many levels. I'm just getting insight from the system about where this – given what I'm trying to achieve, the system's trying to tell me where I am relative to that, and I can decide what I want to work on, when I feel like working on, what I don't feel like working on.

[Music]

Dan Schwartz (18:16):

So Denise, I believe I've come up with a new construct. Well, this is how you survive an academy – in the academy. You come up with a new term. It's called feedback deprivation.

(18:31):

I think feedback is a basic human need. And if you get deprived, it's really miserable. And my example is that earlier today I gave a presentation to about 30 people I don't know. I'm sweating. I'm giving it my whole heart. It's the best thing I've ever done, but it's on Zoom, and nobody gets to say anything. And then the Zoom window closes, and it's over. There's no even, like,  elevator music to allow me to leave quietly and softly. And then, you know, an hour later, I will get an email from the host who says, "Great job. Thank you so much," and that'll be it.

Denise Pope (19:08):

OK. That great-job email is feedback. I mean-

Dan Schwartz (19:11):

No, no, no. They have ChatGPT write it. Because I've done this 100 times, and it's always the same email.

Denise Pope (19:17):

"Great job. Thank you so much."

Dan Schwartz (19:18):

It doesn't matter who I'm talking to. Yeah, it’s "Thank you so much. That was great." Sometimes they use the word “fabulous.” Sometimes they say, "Nice."

Denise Pope (19:27):

So, OK, so here’s my – because I'm right there with you. I think it's so, so hard. It's literally like you're just shouting into the abyss. And so here's my trick. I just need one person. I don't need all 150 whatever. I need one person's face on screen that I can watch, and I can see, and that's how I get my energy. They're my target person.

Dan Schwartz (19:48):

Yep. Hence you decided to have me as a co-host.

Denise Pope (19:52):

Yes, this is why we do the podcast –

Dan Schwartz (19:53):

So you didn't have to do it alone?

Denise Pope (19:55):

100%. This is why we – 

Dan Schwartz (19:55):

Now, I know.

Denise Pope (19:56):

... do the podcast together, so I could see your face, and I get my energy off of your expressions or your eye rolls or your falling asleep or whatever it is. But no, 100%, and I would choose you every time.

Dan Schwartz (20:06):

Thank you, Denise.

[Music]

Denise Pope (20:13):

OK, so I just did traffic school online. [Groans] OK? I'm admitting this to you all.

Dan Schwartz (20:19):

Rolled through a stop sign?

Denise Pope (20:19):

Don't even get me started.

Dan Schwartz (20:21):

Rolled through a stop sign?

Denise Pope (20:24):

No, I was speeding. I was speeding. OK. Uh, traffic school online, right? So, I'm doing it, and I'm answering a question, but they're asking a scenario. "What would you do?" This is not your normal traffic school online. This was a little bit more advanced kind of thing, or it seemed it. They're asking a scenario, and I'm answering what I would do in that scenario. And then if it likes my answer, I can go on. And if it doesn't like my answer, they'll be like, "Let's rewind and try again."

(20:50):

So, how is this different from that? Because I think what you're saying is more advanced than this little traffic school thing that I just took, but how is it different?

Candace Thille (20:58):

There are multiple ways it's different. One of the ways I would actually start with is your motivation. What was your motivation in doing traffic school? Were you actually-

Denise Pope (21:05):

I had to.

Candace Thille (21:06):

...You had to. Your motivation was to finish traffic school so that you get the points taken off of your driving record.

Denise Pope (21:15):

Exactly.

Candace Thille (21:15):

Your motivation wasn't, "Oh! I need to learn how to drive more effectively, so I'm really trying to build some knowledge and skills here."

Dan Schwartz (21:22):

Say, Denise. Denise ... Sorry.

Candace Thille (21:25):

Go ahead.

Dan Schwartz (21:26):

This is why you have kids. Let them do the online school for you.

Denise Pope (21:29):

First of all, don't even laugh. That is illegal. Don't think that we didn't look into this before. You cannot have to . . . You have to swear that you are the one taking the class.

Dan Schwartz (21:36):

Fine.

Denise Pope (21:39):

I know how you're thinking, Dan.

Candace Thille (21:40):

I took online traffic school once, too. And one of the things I also noticed that they did – because what I figured out with the online traffic school – well, it wasn't very sophisticated at all. They told you all the stuff they wanted you to know, and then the next screen they asked you questions basically saying, "Did you read what I told you to read?" They even put nonsense things in there like, "Jenna wears a black jacket," and one of the questions would be, "What color is Jenna's jacket?" They were really just trying to see, did I read it? They weren't really concerned with my learning anything.

Denise Pope (22:12):

Correct.

Candace Thille (22:12):

So, what I did (uh-oh, probably shouldn't say this). Um, what I did was I just got two screens, and I copied all the text off of it because I knew they were going to ask me questions about that page. And then I just searched “Jenna” and found oh, her jacket is black. I never read any of it, but I could answer every single question.

Denise Pope (22:30):

OK, so . . . cheating or shortcut or whatever.

Candace Thille (22:34):

No, it's all about what I started off asking you. What's your goal? If your goal is to actually build knowledge and capability, or is your goal to finish it? And that's why in all the workplace training work that I've done, I never would do compliance training.

Denise Pope (22:51):

Wait, compliance meaning . . . ?

Candace Thille (22:53):

Compliance. Companies are like, "You have to take a class on sexual harassment." That's the classic, right? And it's not that most people – the reason that most people are taking that is because you get told you have to do it, and you keep getting hammered to do it. For most people, it's not because they genuinely believe they need to understand better what sexual harassment is and to stop that behavior and so on like that.

(23:18):

So, motivation is huge in adult learning.

Denise Pope (23:21):

So, you are consciously only taking on projects where you think people will actually want to learn the skills that this technology is going to help them learn?

Candace Thille (23:29):

Yes. And also, I never use as an outcome measure completion.

Denise Pope (23:34):

Oh . . . say more about that. Why?

Candace Thille (23:36):

Because implicitly then the message to me as a learner is my job is to work through all of this learning material and get to the end. So that's my job as a learner. My job as the designer of that learning experience is design something where people get all the way to the end and at the end give it a good customer satisfaction score. "Did you like it, or didn't you like it?"

Denise Pope (24:00):

Or did you learn? Isn't it also, did you learn?

Candace Thille (24:02):

No.

Denise Pope (24:02):

No?

Candace Thille (24:02):

People don't ask that. People ask, "Did you complete it?" Because people assume completion is a proxy for learning.

Denise Pope (24:10):

Oh . . . that's key! That's a key misconception.

Candace Thille (24:12):

It is a key misconception. And so exposure . . . I mean, exposure, completion doesn't . . . like, like your driving school thing. You completed it. What did you really learn except for how to complete that most efficiently?

(24:29):

And when I've worked in workplace settings, I don't do compliance training where the only goal is, we have to prove to people that our employees were exposed to this information. So, when I was building learning experiences in the private sector, I would always partner with business units that had a real business problem, something they really cared about, for which they believed changing the knowledge and capability of the people in that unit would make traction on that problem.

(25:01):

Then you have to build that whole causal chain back to it, and then learners have to believe, "Yeah, I want to build that skill." And then the learning designer's goal is not make something where people complete it and like it, but make something that you can demonstrate actually supported the people to develop the skills and knowledge they needed.

Denise Pope (25:20):

It's cool stuff.

Dan Schwartz (25:21):

How brittle is this kind of technology? Like, just let me take a wild example, that you know a lot about. I'm trying to teach people how to service heat pumps. Heat pumps, right, they're environmentally much more friendly, and so people are switching from furnaces and air conditioners to heat pumps. There's not enough experts out there to teach everybody, right? So we got to go to the computer to train those 800,000 heat pump . . . but it's really physical. Right? Like you got to reach inside and turn your elbow just to the left.

(26:00): 

Can the technologies do this? Or is it just like –

Denise Pope (26:04):

But, I mean, also can't you just learn that on YouTube? Sorry. Not to demean the agent, but, like, right? Are we right back to YouTube here? Go ahead, Candace.

Candace Thille (26:15):

Actually, skilled trades like that, like heat pump installation, there is a real shortage of what we call the climate trades workforce. And there, as Dan pointed out, one of the real challenges is there aren't enough mentors. Apprenticeship is the historic model for learning those kinds of skills. And there just aren't enough mentors for all the people that we need to achieve the climate goals.

(26:40):

So, we are working. We've got a project called HVAC Hero, and we're working with local heat pump installers to develop a blended, computer-mediated human learning experience. And we're targeting accelerating the time it takes to move someone from, "I want to be a heat pump installer," 'til they are at a certain level of customer-ready to go out and do the installation.

(27:08):

From a learning science perspective, it's a really interesting and fun problem too because it's not just cognitive. It's cognitive and physical, and how do you integrate, but it's not either or. It's cognitive and physical at the same time. So how do you integrate paying attention to both dimensions and learning that?

Denise Pope (27:29):

Especially through a screen, which is why I think that's why it's hybrid with a human, right?

Candace Thille (27:34):

Well, you can hybrid with a human. You can hybrid with simulations. You can hybrid with actual fake models of things. There are all kinds of ways that you can incorporate the physical and the cognitive together.

Denise Pope (27:49):

So cool. So cool. Oh, my gosh, Candace, we have learned so much today. I'm excited. Dan, I'm going to put you on the spot, as I tend to do at the end of these episodes. Sum it up.

(28:01):

What do you think you learned about our main question, technology as a game changer for adult learning? What would you say?

Dan Schwartz (28:07):

I think the interactivity is really special. And so, you need an environment where people can take actions, and you sort of want to set them up working on the right actions given what they know, but then you have that challenge of choosing the right feedback.

(28:22):

What's the right feedback at this moment? I think it's an interesting problem. It's something that teachers face. When you're grading essays, there's a million different things you could say. And figuring out ways to get smarter about that, even automate it, save a lot of time for a lot of people and be good for learning.

Denise Pope (28:42):

And Candace, how'd he do? Feedback. Feedback for Dan.

Candace Thille (28:48):

He did great, and I would draw a little bit of the analogy to the homework problem. Just like you can't give the students three hours of homework and expect them to really develop and refine their knowledge, you can't do the same thing with teachers. You can't have them – ask them – to pay attention to 5,000 things all at the same time and refine and develop their skill at teaching. So having the support of the technology that helps both collect the data, model the data, give them the insights so they can make good decisions and refine their teaching.

Denise Pope (29:26):

Very cool. Very cool. All right. So exciting. Well, thank you. Thank you for being here, Candace.

Candace Thille (29:31):

Sure. It was fun.

Denise Pope (29:32):

And thank all of you for joining this episode of School's In. Be sure to subscribe to the show on Spotify, Apple Podcasts, or wherever you tune in. I'm Denise Pope.

Dan Schwartz (29:42):

I'm Dan.