Formative Assessment as Data with Steve Ventura, Ep. 171 Instructional Coaching with Ms. B

Formative Assessment as Data with Steve Ventura, Ep. 171 Buzzing with Ms. B: The Coaching Podcast

Chrissy Beltran  00:00

Spending time in classrooms is one of the most impactful ways we can support teachers and grow our schools. But all of that classroom work can result in tons of documentation that really goes nowhere. If you’ve got piles of sticky notes all over your desk and random collections of fonts and all of your apps, it’s time to consolidate Connect. hub.io is a platform created by coaches. For coaches. It’s designed to help you organize your notes as you meet with educators and get powerful data on your impact in their classrooms. You can also use this information to support their professional learning and growth. Connect hub.io includes great features to help you protect the coach teacher relationship by providing different levels of account access for sharing only what you want with administrators. You can also customize reports to share specific kinds of information with your admin. This is such a great software for coaches. Check it out today at Connect hub.io to simplify your instructional coaching with a free trial. As a literacy coach at an elementary school, I spent so much time trying to find the perfect supplemental resources for my teachers. That’s why I am so excited to share a special resource with you today. decodable texts are essential for students who are working on mastering phonics and primary grades especially, I have come across the sweetest decodable books that I love and my kindergarten daughter loves them to. They’re called Express readers. Each book has a complete storyline includes fun and funny characters like bug and duck. And the stories are engaging and increasing complexity over time as to the words and sentence structures on each page. Sticky words are identified in each book to help children with words they aren’t able to decode yet, or those with irregular spelling patterns. There’s a perfect resource for giving students at your school the practice they need and applying phonics skills, head to express readers.org and visit the Decodable Books page. There you can download a quick keys guide with teaching tips for each book, including teaching sticky word mapping, you can also get free sample decodable sent to your address, head to express readers.org If you’re building your phonics instruction, and you’re ready for some really great readers you’re listening to buzzing with Miss B the coaching podcast where we believe that every teacher deserves a coach, and every coach does too. I’m Chrissy Beltran, an instructional coach, resource creator and coffee enthusiast, and I’m your host, stay tuned for practical tips and honest coaching talk that will help you coach with confidence. Coach, when we think of formative assessments, what comes to mind? Seems like the answer I get most often is exit slips. But there’s so much more to this really important idea. And it’s an important part of this teaching and learning cycle like we’ve been talking about. So today, my guest, Steve Ventura is going to help us dig into the idea of formative assessment, what it is and how we can help teachers use it better. So I am so excited to welcome my guest, Steve Venter to the podcast. Welcome, Steve.

Steve Ventura  03:04

Hello.

Chrissy Beltran  03:06

How are you doing today? Good. Good.

Steve Ventura  03:08

Good morning. It’s great to see you.

Chrissy Beltran  03:10

Yeah, you too.

Steve Ventura  03:11

Thanks for having me.

Chrissy Beltran  03:12

Well, thank you for being here. Can you introduce yourself to our listeners, they may already know who you are, probably. But it’d be great. If you would tell us a little bit about how you ended up doing the work you’re doing, what kind of work you’re focusing on right now.

Steve Ventura  03:23

Sure, it’s Steve Ventura with Advanced Collaborative Solutions. And for the last about 14 years, I’ve been a full time education consultant. I started off Krissy is a very humble PE teacher. And then I got my master’s in special ed and then administration. But things took a really amazing turn and around 2007 or eight when I started working for a company called the Leadership and Learning Center with Dr. Douglas Reeves. And that’s when I really started learning more about what do really good instructional leaders do and what a really good collaborative protocols look like. And then from there, I transitioned to Corwin professional learning, and started working with Professor John Hattie, with the visible learning research. And that really added so much value to all the things we do. I’ve seen you refer to it, too. I know you’re familiar with it. So anyway. And that just led to a book about collective efficacy and collaboration called achievement teams. And now I get to meet great people like you.

Chrissy Beltran  04:30

So nice. Well, thank you. So would the thing we’re going to talk about today is something that I know you’ve written about, and I’m excited to hear your expertise on. Whenever you talk about formative assessment. What all does that include?

Steve Ventura  04:45

You know, there’s so many different iterations of assessment. Everybody has confusion, like a lot of districts I work with, say they have formative assessment, but they’re just three benchmark assessments a year so they’re actually summative. And a lot of times people We’ll get confused with the with end of the unit assessments as opposed to short cycle assessments. And the way I write about it, I think formative assessments has so many different, you know, definitions, but the one I really focused on is giving kids a small set of items to determine how much impact the teachers are having on the learning, and then reassessing that after they intervene with instructional strategies. And so it’s like a book. And for me, it’s like a pre post assessment. But there, it’s very, it’s very finite, it’s maybe seven to nine items of a particular learning target, that teachers want to know, if the kids are actually getting it. And then that information, that data, the analysis helps teachers determine if they need to adjust their teaching, and then classic pre post give it again, to see how much more progress that was made.

Chrissy Beltran  05:52

Yeah, I think that that sometimes whenever we get stuck is we’re not really using formative assessments, effectively to impact our teaching, right? We’re just kind of giving all these tests, because maybe they’re part of our curriculum, but we’re not getting value from it. So whenever we do that, um, you know, we’re talking more about maybe like a summative assessment. Can you talk a little bit more about what summative looks like? Or what an interim assessment looks like? And explain maybe some examples of different types of assessments?

Steve Ventura  06:19

Well, yeah, I first of all, I think I think American students are way over tested, I think they’re under assessed. So summative assessments, again, are usually at the end of instruction Krissy, you know, when you’re done with a unit, when you’re done with the school year, when you’re done with a particular lesson, and then there’s a test. Usually, the difference between that, and interim is that many districts I work when I was a superintendent, we had interim assessments for a year at the end of each quarter. And that was better than just waiting for the state test. But it still wasn’t enough, it wasn’t frequent enough. So then we started going to more short cycle assessments. And that was given really real time feedback on our teaching. So instead of waiting at the end of an interim assessment, or a benchmark, it would be too late to apply corrective teaching, because then the next interim is coming. What I love about the formative assessment piece is that and this is very, very controversial for especially secondary teachers, where formative assessments shouldn’t be graded. Because those results are used feedback to the teacher and the way they’re, they’re teaching. But many times they get graded, and it’s almost academic malpractice, especially if you’re using the data to evaluate the impact of your teaching. But so you have these formative short cycle assessments interim is is better than summative, but then summative assist a state test, you know, Texas has big time, accountability, think time, and everything comes down to that once a year test. And then people are judged by that, once that your test. And I still, I think these more more compact assessments give teachers a better idea about how much progress they’re making, because it may not show up on a state test. But it could show up on these internal local, you know, formative assessments. And so, and all three are fine, by the way. I mean, I give short cycle assessments and interim, so they would be better on the state test. So there’s a nice vertical alignment there. But yeah, the formative is really informing teaching. Yeah, they’ll

Chrissy Beltran  08:20

have different purposes, they’ll have a plane. But But I totally agree, we do way too much summative assessment, and then we’re looking throughout the year towards this long term assessment. But we don’t really even know even if that’s the goalposts, which I personally hope that wouldn’t be but it often, we are not seeing how far away we are from that throughout the year, even whenever people are, you know, trying to use that as like their marker of success, or how do you know how far away these kids are in what you need to do? We’re not adjusting our instruction along the way.

Steve Ventura  08:52

Yeah, and I’m thinking as an instructional coach, you probably see all these amazing opportunities that fall by the wayside. Because people aren’t really skilled at analyzing the data that they have, or even the quality of the assessment is so vague. It doesn’t even give people who give the assessment enough information to determine if it’s, if it’s something that they can actually do. I’ve seen this where I’ve worked with a lot of teachers, they’re very compliant, they’ll do what they’re told, they’re just great people. But nobody gives them enough direction about how to even write an assessment or interpret the assessment results. So it’s, it’s maximized. And sometimes again, the the assessments are so shallow and surface Krissy we could ever even make a determination on for if we’re if we’re actually making some impact there. So very true.

Chrissy Beltran  09:40

Yeah. And it’s not really tied. I feel that a lot. It’s not really tied to what you taught, it doesn’t really match what you taught No, like, like, here’s my learning target. Here’s the lesson where the kids do something unrelated. And here’s my assessment that doesn’t match either of those things.

Steve Ventura  09:56

So you mentioned Oh, go ahead. Oh, no, go ahead. Chrissy.

Chrissy Beltran  09:58

I just was wondering about your short cycle assessments about how frequently whenever you move from worth, you know, just focusing on summative assessments to adding in the short cycle assessments. What does that look like about how often?

Steve Ventura  10:08

Yeah, so great question. So the frequency I always get asked like, you know, in our book, it’s a pre post assessment system cruciate. So here’s the pre assessment here as opposed to assessment. That’s called a cycle. So everybody asks, how many cycles should we do? And my first answer is, well, as many times as you’re willing to evaluate your impact. Don’t try to give a number, you know, and people press like, well, how many times I’m like, Well, certainly not one, but not 10. I mean, and the clients that we work with across the country, when they’re doing achievement teams, or short cycle assessments, they will usually do four to seven cycles a year, okay? And, but it’s got to be doable, our newer clients will try three or four for piloting. And when they start feeling more comfortable at it, they’ll do more but but, you know, a lot of the information we got about these pre and post assessments came from James poplin, who wrote a book called teach better test better. And he’s a retired professor emeritus from UCLA. He is the one that sparked my interest in these pre post assessments, guys, a psychometrician. He knows a lot about assessments. So sometimes it’s not how many you do Krissy. It’s the quality of the items. And the data that they generate. Three or four could be amazing if it’s really good quality information. So I mean, remember, those assessments are only as good. I mean, the inferences we make are only as good as the assessments we we administer. So. But, again, I mean, you know, there’s internal formative assessment to where it doesn’t have to be pre and post just teachers give informal assessments all day. But this would be like for a horizontal team, like, Yes, three 3/4 grade teachers who want to give the same pre and post and work together as a, as a horizontal, collaborative. But, but that’s how we usually use it more

Chrissy Beltran  11:56

of like a common assessment, so we can all get together. Yeah, and look at data together, have a conversation about what we see happening across our grade level, and then share, you know, ideas and strength. Yeah, hey, oh, this is something that’s working for my kids that you can try out and ask questions and really have that healthy?

Steve Ventura  12:12

Yes, exactly. So I You didn’t even say common formative assessments, but that, but they usually are. And that’s how we use them when we when we work with our clients schools. But I think when you know it, depending on the the mindset of the people we work with, I mean, 80 to 90% of the clients, we have see the value in this because they for the first time, they can slow down and go deeper with fewer learning targets, instead of superficially covering everything, and then making some amazing determinations about what is the next steps for progress for students, you know, and in testing, there’s, there’s, there’s achievement and progress, both are very important. But sometimes when kids make progress, they may never be in a student, but they’re progressing. And I’d rather have kids who could progress more than one year and one instructional year, then just, you know, know how to play school and get A’s over and over and over again, not not diminishing that either. But I think these formative assessments are designed to accelerate learning. And we have a lot of kids because of post COVID, that really needed double the speed of progress. So there’s so many advantages to it.

Chrissy Beltran  13:23

You know, you mentioned you’ve mentioned grades a couple of times, and I’m glad that you brought it up, because I actually spoke to a coach recently. And this is going to be an episode on the podcast that’s coming out the week after this one where she was working with teachers, and they were saying, but I take I already have grades in my gradebook, why do I need to have assessments on top of the grades? You know, what, what is the difference, and they couldn’t understand the difference really, between the assessment that’s focused on specific things, because I’m thinking you probably bundle them alongside like units or chunks of standards that go together, and then you’re probably organizing it by standard and like specific skill rather than an overall grade, which could be about a lot of different things.

Steve Ventura  14:05

So it’s a misconception, right, so grading really is just an announcement, like, here’s what you’ve done, those are summative grades, by the way. So um, the the ingredient is the most emotional, one of the most emotional topics in education right now. Right? So a lot of times when I work with teams, and they get pre and post and I say please don’t grade the pre assessment, but you know, what, if you’re going to teach and then give the post assessment, okay, you can grade that if grade ain’t your thing. But you know what kids need more than grades. They need feedback. They need feedback that’s accurate and fair and timely and understandable. And students need to be understanding that not everything they do should be graded. And the problem is, is that teachers grade the way they were graded. Yes, so everything gets a grade and then know what happens. Kids don’t want to do anything. Because they want to know if this counts. And then we have to go through all that. Like everything in my class counts, like if they don’t see a letter grade with the assessment, they they lose it. As a matter of fact, I think Dylan Willam is an amazing author. And I think he wrote an article called inside the black box. And he had this one thing that was so interesting. He said, Look, as you want to know how dependent kids are in grades, let’s say Krissy, you have 30 kids in your class, they all turn in 30 assignments, half of the assignments, you gave them a grade and feedback, right on the paper, right and feedback, the other half of the kids didn’t get a grade, just speed back. And you watch what happens between the 15th and the 15th. The kids with the grade will not read your feedback. They’ll only look at the grade and put it away. But the kids that just have feedback and no grade, they’re going to ask you like, where’s my grade? And you will say things like, well read my feedback. You because so they’re so programmed, this is why teachers get frustrated. Yeah, what I mean, normally, I would be using the pre assessment, as nothing more than feedback directly to the students. So they could do error analysis with those, those results do some really good, amazing, engaging instruction between the pre and post. But I would just tell people who are really embedded in grades like if you do the cycle, then you can you have enough information to give a grade the second time. But the first time, we’re really using that to make inferences about our practice. Interesting is so emotional, would you agree? Oh, yes, I

Chrissy Beltran  16:32

agree. And people are people are very feel very strongly about grades. And that’s one of the issues with grades is okay, you bring your gradebook, I bring my gradebook. And we look at, we’re looking at two different things, even if we grade the same assignment. Because open ended assignment, even if we’re using a rubric, I mean, you really have to be calibrated to use a tool like that in the same way. So we you can’t use grades as as data necessarily, for that purpose to see where you are. Because we’re all doing different things with grades.

Steve Ventura  17:01

You got it. I mean, I think the same work by the same kid ought to get the same grade, even if the teacher is different. And people have a hard time coming to consensus. What’s a D in your class could be an A in my class. Yeah. And I just think that’s unfair to the kids. Yes, it

Chrissy Beltran  17:17

is. And that’s a whole other issue. But that is why those are not assessments that we can use as common assessments. Like they’re not, they’re not really formative either. Because what are you looking at? You’re looking at maybe a, you know, project that involves lots of different things. But unless you analyze that by little skill, it doesn’t tell you what to do next. Exactly,

Steve Ventura  17:36

exactly. So and these short cycle assessments are really designed to really determine what skills and concepts the students actually know. You know, and you know, what I’m talking about just like, what’s the verb and the standard and the noun phrase like, that’s what we should be assessing, like, determine or analyze or summarize or explain how. The other thing with these assessments Krissy, when we do this work with them is that they’re very aligned to Webb’s Depth of Knowledge. And we have to show teachers that certain items are assessed certain ways, like do k one verbs, true false, multiple choice, but do k three verbs short, constructive response, non traditional selective response, where there’s more than one answer? A lot of the clients we’ve worked with didn’t really know the difference between a lower level verb or a higher, more complex verb. And so then we show them item writing guidelines, like, you know, do K for Chrissy can’t be measured with true or false, it’s impossible. Right? It’s so complex. So that’s another thing that we focus on with formative assessment. And we always tell the teachers like, do K levels are not enough for kids. It’s not based on what we give only do k one test to kindergarteners, and then it goes up as a grade levels, they’re not developmental, All kids should be exposed to that level of complexity. So, but, but I really like when we show teachers like, you know, there is a strategy for assessments. And you know, what happens? Then they look at the assessments they’ve been giving, and they look at our guidelines and say, Wait a second, I’m using the wrong format for this item. And they make a couple of minor changes, and then all of a sudden, if they do create their own assessments, they’re very valuable, valid and reliable. I mean, so we I don’t remember ever taking a class before I got my teaching credential on how to write really good formative assessments.

Chrissy Beltran  19:35

No, I know I don’t they didn’t have a class on I know that it was a part of a class or two, but I don’t know how good it was. But I but the actual like creation process, I probably got to create one as part of like a other projects. You know, you have to write a lesson plan and your lesson plans are 75 pages long in the college and then one piece of your lesson plan and so it’s so removed from the reality of the class. bathroom that it wasn’t very impactful. Yeah.

Steve Ventura  20:02

Well, I didn’t know what a question stem was. I didn’t know what distractors were. Yeah, isn’t it? So I started getting into that. Then we started helping people design high quality assessments, like all answers should be plausible, but there should only be one right answer. Well, what we see is really random selection processes for students. The other thing is that, you know, the students shouldn’t be able to easily eliminate any of those choices, they should really be deep fighting, like, which is the best answer. So there’s so many valuable things, you know, we can do with these, with these formative assessments for sure. And just showing people item writing guidelines, like what’s the best way to assess the verb, identify? or analyze, you know, what, what are the best, you know, ways to write that assessment? Yeah,

Chrissy Beltran  20:55

I think you’re, you’re right in that. And I’ve worked with a lot of teachers as a campus coach, creating our own assessments. And sometimes the the verb is not aligned, like we talked about earlier, you have a learning target, you have a lesson, you have an assessment, they’re not aligned to each other at all. We’re talking about describing in our target where you’re actually having kids maybe identified during the lesson, and then the question is asking them to explain something. And so we’re all over the place. And we don’t really have good alignment so that we’re not teaching or assessing what we think we are. Or

Steve Ventura  21:26

you have it, right. Yeah, I mean, that’s an example of what I called deficit instruction, where the standard might say, explain, and analyze, but the teachers are assessing whether they can identify and find. And that’s two levels of complexity below what those actual skills are. So we have to be very careful that we understand what skills and concepts were assessing, and then the items are completely matched to that, and then you know, like, then you have something that’s valid and reliable. And then teachers can make super, super informative inferences about where the students actually are. And I think, just to say, I think formative assessment is the best way to overcome some of the issues we have with COVID. I think we should be doing more of it and less testing, and just assessing kids and in chunks along the way where it’s more frequent. And then the teachers get way more timely information. Before it’s too late, right?

Chrissy Beltran  22:30

Yes. Oh, yeah, absolutely. Because we hear a lot of, you know, there’s a COVID learning gap, kids are very behind the kids don’t have the skills they used to, but then whenever we get right down to it, they don’t always know exactly what they do have and don’t have, because it’s all really very overwhelming. So yeah, if we can get some better information about where our kids are along the way, then we can maybe target those areas and those students who need certain things more. I had a teacher once told me, I’m targeting everything. Target works. Well, a little bit about, you know, well, what do we what does it mean to target something? And why are we talking.

Steve Ventura  23:09

And, you know, Chrissy, and other really, really amazing thing that happens with formative assessment system that teachers become better consumers of writing learning intentions and success criteria. Teacher clarity has a point 775 effect size, you know, when teachers write success criteria and share it with their students, that’s essentially the assessment. They have literally written the assessment, and then that’s what they teach the kids to do between the pre and post. So if the kids can go through that three to five or six criteria for success based on a single learning target, they’re going to, they’re going to crush their post assessment or any assessment if there’s a little bit of pre teaching before they give the assessment. So you can see, like, I know, you’re familiar with this, too. But you can see what a great segue. You start off with learning intentions and success criteria, and then you assess those, it’s completely aligned. So it’s good way.

Chrissy Beltran  24:02

Everybody knows what everybody’s trying to learn, right? The kids know, the teacher knows, and they know what it’ll look like when they have learned it. Like those PLC questions, and then we can actually figure out okay, now that we know who knows it, we know who doesn’t know it. And we know what we need to work on next, because it’s all Yeah, that’s it. So what are some of the other common mistakes that you’re seeing whenever teachers try to implement formative assessments because we’re talking about a lack of alignment here? And probably not frequently enough? So what other issues are you seeing pop up? Um,

Steve Ventura  24:33

the length so sometimes, what differentiates a formative assessment to a summative assessment is formative assessments are more about content chunking. Like really, really going from a floodlight to a flashlight like I really want to assess this single skill. And but teachers have the curse of the conscientious. They’re so professional, they want to assess everything. All of a sudden, the formative assessment turns into more of like a 40 question test. And so we see people trying to do too much with formative, like, it doesn’t need that many items, especially if you want to see if the teaching was impactful. The other thing is that the assessments that are given, again, are misaligned. These are some common misconceptions about this. And then a lot of people believe that the only way you can assess kids, even with the more complex verbs like Dlk, threes, and fours is through short, constructive response, that’s certainly the best way. But if you write really good question stems and multiple choice items, you can still be extremely effective, even assessing more complex verbs for skills. So these are some things that we go over again, with people like listen, we know that we need kids to do short, constructive response and extend a constructive response. But you can still assess those areas with some really well written selected response items. So there’s a little myth that you can’t really assess those with anything else, but writing, but you can, if you if you’re thoughtful about how you design them. So I think the length is an issue. And sometimes I think, the fact that we can use selected response or matching or true false even with do case, twos and threes. Kids can still write answers, but you can still assess them with some pretty good accuracy. And especially if you don’t have a lot of time to score Krissy. Like the teachers I work with, like, listen, I know they need to write, I get it. But if you want to get results back right away, I don’t want you to spend your whole evening scoring these really long assessments, like it’s got to be that you can just look at them and say, boom, that’s the score. So

Chrissy Beltran  26:49

yeah, yeah, I’m also glad that you brought up a few minutes ago, the plausibility of answer choices. You know, I remember whenever I first started teaching, there was often an answer that was just not even in the realm of reasonableness. On the big test, you know, whenever I started teaching was the first year of tax. And in Texas, okay, so I took toss as a child tax was what was in place whenever I started teaching. And then towards the end of my career with star, which is currently what is what they were they have star, these newer, more modern tests don’t have ridiculous answers anymore. They are all reasonable that all are related to the texts are all from the text somewhere, if you’re talking about reading, specifically, they are related to that topic. They’re not You can’t look at something and go, Oh, that’s not even about the same topic, mark that out. It’s it connected. So kids have to really understand at a deeper level. So whenever teachers write answer choices, they instinctively want to put in a couple that are ridiculous, that are throwaway answer choices. They’re not, they’re not there anymore.

Steve Ventura  27:48

So I just, that’s the research we’ve been reading, like, don’t waste a choice, like the three are there, and then one’s completely supposed to be funny. Like, or, and the kids can literally eliminate that. And now they’ve already increased their probability of getting this correct, which we want them to be successful. But they still have to, they still have to lead through all four possibilities. And then see which one is the, which is the best one. But

Chrissy Beltran  28:16

almost be better, just to give them three choices than four, if you’re gonna do that anyway, you know, yeah, it doesn’t really add anything to what they can do. And it doesn’t matter the assessment that they’re going to see down the road.

Steve Ventura  28:26

Well, you know, another really effective item to use for our formative assessments are the matching, where the premises, like say, there’s four premises, but there’s five responses. So there’s the students, instead of just four to four matches, yeah, more responses than premises, and then the hits, again, really happy to think clearly like, one of the responses that I’m not going to be able to use. So you know, it’s okay to have form for like, here’s my premise for statements, here’s four answers, just draw a line matching, this is the way I went to school, right. But now, all of a sudden, when teachers add more responses to the premises, the students have to really think, clearly, like one of these doesn’t go with the other four. So there’ll be four, but there’s an extra one. And I love that process. Because when the premise column has less items in the response column, there’s some extra thinking happening.

Chrissy Beltran  29:25

Yeah, for sure. I love that you’re reminding me of something that I used to do with my teachers whenever we talked about, like making inferences in a text or something along those lines, and we would have text evidence on one side, yeah, inference on the other, um, little cards. And I did also include extra, you know, choices that didn’t match anything. And then kids had to go through and say, which evidence is supported, you know, which inference is supported by which evidence and they would match it? Yeah. But uh, thanks to know that that’s something that you’ve you’re out there doing, too, that that was a good practice.

Steve Ventura  29:53

Yeah. And I’ll go back to your question. Like one other misconception or common error is when? When kids don’t know any of the answers Krissy? I mean, I’m speaking from experience, we tend to focus on the longest one, what’s one sticks out the most could be the right answer, right? So with formative assessments, all of the all of the choices need to be equal in length. And that way, it’s better. But I just read some research where teachers, unconsciously, I mean, not unconsciously, subconsciously, want to make sure that the answer that’s correct is clear. So they’ll pick it. So they tend to make that one the longest. And the kids are looking for the longest one. And so now they’re all together on the same team.

Chrissy Beltran  30:43

It works out

30:45

for what you want. Yes, really? It’s really interesting. That is interesting.

Steve Ventura  30:51

Honestly, the it’s super interesting, of course, if this is the whole thing was put all together. So the longest answer is C. This is everything that no, because the most common selection is C. So now the teacher made the correct answer is C, and it’s the longest one that sticks out the most. So equal in length, distractors, really add more quality to the assessments again, so we’re talking about like, like premise mismatch items, these item writing guidelines, and a lot of teachers will tell me like, I never knew this rules about writing assessments, like all of the choices should be in alphabetical order. So the kids can’t find a pattern. So really interesting stuff, you opened up a couple of little things I never get a chance to talk about, but you I love it.

Chrissy Beltran  31:42

I love to get into the details, because I really, it’s strange, but I enjoyed writing. As a campus coach, I feel like I spent a fair amount of time writing assessments, helping teachers look at the big summative assessment and working backwards from really analyzing it. And I feel like you’re right is you don’t really understand how that test works and how its structured and how the choices work. Unless you’re trying to create something that does a similar user that is written in a similar structure, but for a shorter period of time and covering less content. Yeah,

Steve Ventura  32:15

so so funny, crazy, like, I never thought my pathway would be really getting into the formative assessment piece. And through, you know, just reading and finding these amazing authors that are out there like, like, again, James pop them. We had. You know, our book has been published with ASCD. So I get a chance to rub elbows with some really amazing people. And back in the day, when I was first teaching, I never thought it would be that interested in assessment creation and helping teachers use the data to assess their impact. But it was just, I was so very fortunate again, to learn from John Hattie formative assessment has a very high effect. And to see its value is something that sometimes schools don’t stress enough, because there’s so many things on teachers plates, that, to me, this would be one of the top three, like we really have to have a good formative assessment system in their building it and it should prepare kids for a larger assessment, but it shouldn’t be too much for the teachers where they start making when do I have time for this? That’s the reason why. Sometimes we don’t get into it. Yeah, the teachers time is completely blasted out. I mean, I’m always telling leaders that want me to come work with them on achievement teams and assessment. I’m like, Listen, I want to come and I’m going to help you. But you know, for every new initiative, you ask teachers to implement, you should be taking three things off their plate. That way, they feel like, okay, we have time, or just put them in timeout. But what happens? Is it just another one on or another one and another one. And they’ve learned something about teachers Chrissy like when it comes to things like you have to write more formative assessments. And they ask why do we have to do this? It’s not because they’re being unprofessional, but they’re really worried about is when am I going to find the time to do this? And so it comes out like you think they’re being difficult, but they’re like, we’re up to our neck right now. With everything so

Chrissy Beltran  34:19

anyway, yeah, no, it’s absolutely true. And I’m glad that you mentioned that because as schools are trying to implement more formative assessments over you know, throughout the year, it can be a real challenge to figure out where those those are gonna go and how you’re going to meet with your people in a timely manner, not only to create them, but to look at them and then do something with that information or else it’s pointless.

Steve Ventura  34:40

Exactly. And I think again, I want I’m so enthusiastic about it, you actually see the value I know you can appreciate it, but once we unlock unlock that learning for teachers about the how, how amazing the assessments can be if they’re written right and we’re Can we assess them correctly? Literally, teachers begin to feel more success. And it’s contagious. They want more. So the more we actually learn how to use this process this, whether it’s a short cycle pre post or just as just an assessment, but it’s common among the grade level or the content area, so much value there, yes. Where people can under understand that the process was never designed to make it more difficult for you to be a teacher, it was to actually help you be more effective. So but I think, when I asked teachers, what’s the reason to assess what would be their top three things. And my top three reasons to give assessments is to evaluate our effectiveness as a teacher, to provide feedback to students. And then to identify what the kids need the reach is keep those three things in mind, then you have it, but you don’t use them for grading, you don’t use it to evaluate the quality of your curriculum, you won’t use it to revise your goals, it’s literally in any order you want. I mean, it could, it doesn’t have to be one, two, and three. But to me, number one, just because of my visible back visible learning background, would be to evaluate how much impact we’re having, then we give the feedback to the students and identify what they need. So if you have that mindset, it’s it’s less the way to get in to more assessments. But if it’s just about grading, prime will be very successful with the way we’re administering the assessments that we think should have the greatest impact. So again, you said it like there’s there’s summative, interim and formative, we’ve been finding the most success with teachers who invest in the form of that. And I think they just become better consumers of how to create the assessments and then use the results.

Chrissy Beltran  37:00

Yeah, I think you’re absolutely right in all of that, because I’ve seen that happen. You know, whenever I started out as a new coach, the campus I worked at did not have, they’d started to give assessments, just because that was the that was a new principle the previous year. And that was the first thing she could get them like, this is part of what we’re going to do, we’re going to start to give common assessments period, the quality wasn’t there, they weren’t super, you know, organized in terms of what they were teaching curriculum wise. So then it didn’t always match. So it was little all over the place. But she was like, we’re trying to build a habit. And we can make a habit better, but we can’t do it. Well, if we don’t do it at all. So we have to start somewhere. And then they got used to sitting together in data meetings, looking at the assessment. And over time, you know, my year, so that was the year before I got there. And then I was there for five years. And I saw Lea the way that we built assessments, the way that we understood them the way that we use them, teachers were like, Okay, let’s plan it in, we planned in the assessment date at the beginning of our planning. And then we you know, work backwards from that we knew when we were going to be wrapping up our units. And we knew what was going to be on that assessment because we had written it together. And that was such a huge change. And by the end of my time there, that was just a regular part of their teaching. They were used to it, they knew what was going to happen. They understood how to look at the data and pull it backwards and say what does this mean about what we’re doing and what we need to do for kids. And it was like, a huge shift, but it takes time to get there. Yeah, it doesn’t happen. I mean, even the first two years, it’s just such a change, because like you said, teachers are barely treading water. So whenever we put anything new in place, it’s like, Oh, just a year of just getting used to it.

Steve Ventura  38:42

Yeah. Well, and you feel like that we have so much in common? Will we have time to do the research for teachers, like what’s the power of assessment, teachers do not have time to go home at night, and read a book about assessments, like you said, they’re treading water, they’re doing the best they can. And so people like you, and I can come in and support them and tell them that this is the actionable thing that can happen in your class. And I think we should make it as easy as possible for them to give these assessments. So I have all kinds of templates and tools where it’s just like, let’s design it. But I just don’t, I just don’t think that teachers have an opportunity to read the most contemporary research about the value of assessment. So not that they wouldn’t want to have time. I just think that look at all the things that are on their plate right now. I mean, it is a it’s been challenging post COVID. And I think a lot of teachers are frustrated, but the with the slow progress kids are making. But again, this would be one strategy that could considerably accelerate student achievement. So see how they, how we get into it.

Chrissy Beltran  39:50

So whenever you talk about formative assessment with teachers, what are you what are you talking about with them? What are you doing to kind of help them understand the process? You mentioned some of the reasons starch and you know, actual assessment writing? What other things do we need to be bringing to the table?

Steve Ventura  40:04

We, you know, it’s funny, I never make any assumptions about Webb’s Depth of Knowledge, because sometimes I’ll just start saying things like D. Okay, let’s code your standard. Let’s see how rigorous it is. And I had one really humble teacher come up to me in between sessions and say, Steve, you keep talking about DL K, what is that? And this is, what a great way to just come in ask I’m like, This is why I never assume anything crazy. Like, I’ll do a whole section on how to use there’s 57 Hot verbs from Webb’s Depth of Knowledge. Many times the teachers aren’t really exactly sure how to use them. And so the thing we start off with is what is the, um, the way to determine the cognitive demand and rigor of whatever you’re assessing, and we start there first, we can get people to be more comfortable with the Dlk verbs, then the assessment, item, writing guidelines are better. And like I said, before, I’ve had so much opportunity to read what’s really relevant, you know, when we get into it, there’s just just an amazing amount of information that you would have to just really focus. But I think I started off with levels of rigor. First, we, through many teachers like my age that started off with Bloom’s Taxonomy Chrissi, which is six domains. Yeah. And then, then in 2010, at least in California, my home state when Common Core started, kind of being assessed when everybody recognized like, wait a second, this is going to do K levels now. And you know, the difference between blooms and Dlk is one’s more about difficulty. One’s more about complexity. But when I talk with teachers about formative assessments, we really want them to have a solid understanding those four levels of verbs, what do they really mean? And then what are the items that go with that level? How would you assess those items? The other thing when we talk about this is I show teachers a lot of samples of formative assessments that I’ve written for other clients. And why don’t right away they see the value of these items. I mean, again, we give them really good item writing guidelines. But at the same time we show teachers like, even if your kids choose the wrong answer, you should be able to make some really good inferences because you put incorrect answers, that would be common mistakes for kids. So they’re all there by design. And you know, who’s really good at his math teachers? Those have multiple choice? Yeah. And they’ll put in three different ways they know the kids will miscalculate. And if all of a sudden, they already know how to how to correct it, how to fix it. So those are some of the tools that we introduced. But I think the biggest thing for us is to make a compelling case, why would you do this? And I think at a professional respect for teachers, Christmas Krissy, I think we should tell them why before how. So anytime I do assessment training, we have to talk about the why First, give people a really compelling case to take a look at this. So funny. You’re bringing this up next Saturday, a week from today, I’m working with a district on a Saturday, they cannot meet any other time. And they want to learn more about writing assessments. And so you know, prepping for that they’re an amazing district with amazing teachers are coming in on Saturday, I know they’re getting paid not enough. But we’re going to write them, we’ll spend six hours that day, and I’ll show them how to write the assessment. At the end of that segment, they will have a product, it’s a production day. So it’s not just the theory of assessment is like here’s how to write one. Now go ahead and pick one of your standards and write seven to nine items.

Chrissy Beltran  43:46

I love that. It’s got to be relevant and useful. What are they walking away with? They still have to go do all the work.

Steve Ventura  43:52

Yeah, I mean, they can they have item banks Krissy. Thing is like, yeah, I always tell people use your item banks, you don’t have to start from scratch, just use the method that I’m showing you to determine if those item back items are really valid and reliable. I’ve seen really horrible written assessment items when purchased item backs. Yeah.

44:13

Right. Yeah. And even

Chrissy Beltran  44:15

just student materials that are supposedly aligned to Star Line to whatever, but they’re really not you look at it, you know, it’s never assessed this way. It never know like this, and it doesn’t even make sense to assess it in this way. And

Steve Ventura  44:26

the thing is, is like there’s no, there’s no quick fix. I tell clients again, like there are certain websites that don’t screen the content very well, but they’re for sale. And teachers are looking for a quick purchase. I gotta find an assessment. And those items haven’t been vetted that these items look really nice. They have really cute Elementary School font. They’re very colorful with images. But when you look at the quality of those assessments, they’ve made almost every mistake you and I just talked about to avoid Yeah, when they’re selling them and people are buying them up, I see them all the time. So,

Chrissy Beltran  45:04

yes, same. There’s got to be a way, right, a lot of questions, and we would like tape over it with that white tape. And then I type out and print out, you know, for the master copy, like what the proper questions should be and ridiculous and we and how much money was spent on those items and who chose them? Right.

Steve Ventura  45:21

Right. So, but whether people are writing them and selling them, or we buy them from a publisher, still, I think that teachers should be very good consumers about determining whether or not the assessments they’re using are valid enough so they can make some pretty good inferences about the results. Yeah.

Chrissy Beltran  45:48

Yeah, they start working out because then it’s not you’re sitting in a PLC data meeting with teachers. And you’re like, Okay, well, that yeah, you’re right. No, that one didn’t make sense. No. Oh, yeah. No, you’re right. There are two right answer choices for that one. Oh, yeah. You could support either vote. And it’s just terrible. And it’s a big waste of everybody’s time. And you wasted the kids time as well, because they took a test that meant nothing. And then they worried about it. Exactly. Yeah. It’s it’s a bad process. So what are as you’ve worked with different grade levels and subject areas? What are some formative assessments that you’ve used? You talked about true false and matching? And I love that. Are there some other things that you’ve used that work? Well, with different grades? Like I’m thinking of very early childhood, their assessments have to be developmentally appropriate? Yeah,

Steve Ventura  46:30

the developmentally appropriate are usually with images, their visual assessments. I, when I work with kindergarten teams on short cycle assessments, sometimes those assessments may only have four items. But the teachers are showing the students how to select the best site and just circling it, like their images from the book, right? So they may not be able to read the question stem, the teachers, of course, read that to them. But they can detect they can make, they can do things like compare and contrast with images. And you know, that the knowledge level three, that’s pretty good for kindergarten. The other assessments we use, especially for, say, the specialists, like band, or art, or other non traditional assessments is to help the teachers create authentic performance tasks where the kids perform the assessment as opposed to paper pencil. So they have certain things they have to demonstrate, like, especially in 3d art, I mean, they may not take a paper pencil test, but they will make or create something with a rubric. And then they can be assessed that way. And I feel like between very traditional formative assessment, but creating assessments that are based on performance tasks, you know, each task builds on the previous tasks. So they might do the first task, the second, third, and fourth, or they’re leveling up from Dlk, one to two or three wherever it goes. And so sometimes, it’s literally them creating something, as opposed to a paper pencil assessment. So you can use those authentic performance tasks to gauge just as well. Plus, you can use that in lieu of paper pencil for kids who struggle with some comprehension and reading ability, they can still demonstrate proficiency verbally or through some sort of tasks. So it’s really it’s so many different options. But in lower primary, a lot of times the teachers automatically will assess things like letter recognition or number recognition. And those are fine. They may even stay on reading for foundational skills, and I just try to gently nudge them and say, Listen, those are all great things. You’re going to assess them anyway. But there are standards in kindergarten where they do have to compare and contrast. Yes, characters, settings and events. That’s a great assessment. Because they need to do that between now and their senior year. So usually, when teachers asked me, Well, what do I how do I decide what I want to assess? I’m like, Well, if you have two targets you’re looking at pick the one that’s more rigorous, and assess that. So I feel like lower primary pre K TK K, they can still determine and tell you, what’s the same, what’s different. And I think those are the ways that we could assess all kids until they have equal access to really high quality assessments, they get a chance to let the teachers know how much they know about those those topics. So the older they get, the more traditional the assessments are, like, could be paper pencil, but I still like to have a variety of items on one assessment, multiple choice, true false, short, constructive response. Just have all three, right? That way that let’s say a teacher designs, a big essay, and the teacher and the student didn’t study for a big three paragraph essay. And that’s all that’s on the assessment. Well, then the whole assessment is, is scuttled, because they don’t know it. So I was putting other items for them to demonstrate proficiency in case they can’t answer a five point essay, but they still can demonstrate other things. So usually, really good assessments Krissy have a combination of matching true false selective response, fill in the blank, short answers, and so forth. And then the teachers have a lot of really good information to make really good inferences. But you know, you have blue and say, just do a 50 question true false, you know, you could have different items in there. And that’s typically what we recommend, as the kids get older, like, giving them more than one way to demonstrate they know your content. And so, sometimes the big mistake is just to have them essay stuff. But again, I mean, sometimes a student’s senior year could be functionally over if they only were being assessed on something that large without any other items that they could demonstrate that they, they knew the material. So

Chrissy Beltran  51:06

yeah, that’s a great point. Because you’re scaffolding it at different levels in your instruction, we would want our assessment to have different levels as well, so they could demonstrate understanding, and we can see where it starts to break down.

Steve Ventura  51:18

Yeah. So those are some of the things that I you know, when we, when we do the the training, that’s got to be so appealing to teachers that they would want to use it. So again, as I mentioned before, like, we provide a lot of templates, a lot of clarity. A lot of reasons why and help teachers actually write the way they are explicit what the students have to do. I feel like the more explicit teachers are, they will get better results. We tell teachers that who teach older kids, that if you really wanted clarity on an assessment item, you would specify the point value, right? The acceptable response length, and how much time they have, because kids ask that every day, how much is this worth? How much time do we have? You know, how long is it supposed to be? So a lot of times with these shorter, extended constructive response, I think, if we give them those the three things like well, it’s worth 20 points, it should be 200 to 300 words in length, and you have 20 minutes. Go ahead and start. You know, and I think that’s so suitable and specific, that the kids will know exactly how to continue to move on with that with that item.

Chrissy Beltran  52:41

You’re telling them what’s expected of them. Yeah.

Steve Ventura  52:46

But my thing Krissy is like I think we should be a ploy, we should employ more questions that have shorter answers, rather than just a fewer questions with longer answers, like I mentioned before, because that’s better content sampling from them. And then we’re trying to increase the chance that they actually do well, by giving them those multiple opportunities. But if a kid hasn’t really properly stuttered for the content for one big three item essay test, then 1/3 of that assessment is just really completely, as I mentioned before, scuttled so trying to do better content sampling, then the teachers get a better idea of what the kids actually knows just saying, Well, they didn’t know anything. Right? They probably do. It just was the way the items were formatted.

Chrissy Beltran  53:34

That makes so much sense. Thank you so much for sharing everything that you have about formative assessment. I know this is something people are undertaking, and it’s a big one. You have to have, there’s so many moving parts to make sure that it goes well. So I’m really appreciate everything you share today.

Steve Ventura  53:48

Thank you.

Chrissy Beltran  53:49

Yeah, I get now to ask my favorite question. It’s, it’s what is your favorite thing right now. And it can be anything a product, a TV show, anything you love. On the start, I

Steve Ventura  54:00

would prepare for this. So first of all, start off with a favorite favorite book I’m reading today that goes along with what you’re talking about. So I’m a big fan of Karen Hess. And Karen has just put out a new book called breaker by design, not chance. I mean, if she is completely amazing, and if anybody was wondering how to take more actionable instruction through assessment, this is the book again. It’s IT people may not have time to read all of it, but it’s the type of book that you can go to and find out more. So it’s, it’s all about scaffolding and helping teachers design complex tasks. That’s one thing I’m reading right now. I love it right? It’s helping me tremendously. And then non education. We just brought home a brand new Golden Retriever puppy that I am surprised you haven’t seen it running.

54:55

Yeah. That’s great.

Steve Ventura  55:00

My that has been amazing. We’re dog people, we we had to put a one of our other retrievers to sleep about six months ago. And so we were waiting like, Well, how long can we go with that another retriever because we love the breed. So it’s been a blast. Raising this little puppy, she’s just so cute. And then the other thing I’m really enjoying is just like the the time that I’m home now I get back, I get to get back into a little bit more of my exercise routine. And I started doing spinning classes. Chrissy Are you aware of these classes? Yes, I am, where you have that one caffeinated instructor in front of the class with 20 people riding bikes. And when I took that, uh, you know, there’s a lot of sweat involved with spinning Crazy, right? Yeah, like, my arms hurt. My legs hurt my thighs hurt. By the time I got those spandex bike shorts on I missed the class. Believe it. I don’t know if these jokes work for everybody out there.

Chrissy Beltran  56:03

I’m sure they’ll get up because yeah, that’s I think that’s a common experience.

Steve Ventura  56:09

Anyway, that’s what I’ve been up to. I’m just been enjoying. We’re so grateful with our relationship with ASCD. So things have been going pretty well.

Chrissy Beltran  56:20

All right, well, great. Well, where can people find you online to learn more?

Steve Ventura  56:23

Just my name Steve ventura.com. And, or they can do a search for achievement teams, but just my name, Steven b.com. And then they’ll can see all the resources we have up there. All

Chrissy Beltran  56:34

right. Sounds good. Yeah, good luck raising that puppy. Because those are a lot of work.

Steve Ventura  56:39

They are right now.

56:39

Lots of energy. Yeah. Tons.

Chrissy Beltran  56:46

All right, Coach. That was the great Steve Ventura. So much excellent information. And I love it. Whenever our guests get into the nitty gritty and talk about the specifics. He gave us so many things to think about as we are working with teachers and planning out what these assessments look like how we’re going to use them. I just thought it was excellent. And, you know, I, I guess I haven’t talked about this in a while. And I was really excited to have this conversation. So I hope that that came through today. Next week, we’re chatting with a coach in the field about how they can better use data and assessments with their teachers. And this is the one that I mentioned where they got the question about, well, what why are we why are we doing this? Whenever we have grades, right? Why do we have to use assessments whenever we already take great. So we talk a little bit about that and a little bit about some of the other things that they have had to overcome as they’re working with data and assessments. If you are using assessments, regular frequent common assessments with your teachers, and you want to start having better conversations about them, you can go to buzzingwithMsb.com/Episode 171 and scroll down grab the data bookmark because it has a series of questions on there that will help you have better dialogue with the teachers about their assessment results. So next week, again, you’re going to join me for a coaching call where we problem solve a coach’s work at school and until then, happy coaching Thank you for listening to buzzing with Miss B the coaching podcast. Want more coaching ideas? Check me out at BuzzingWithMsB.com and on Instagram at buzzing with Ms B. If you love the show, share it with a coach who would love it too. Or leave me a review on iTunes. It’s free and it helps others find this show. Happy coaching!